whitespace-format 0.0.3__tar.gz → 0.0.5__tar.gz

{whitespace_format-0.0.3 → whitespace_format-0.0.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: whitespace-format
-Version: 0.0.3
+Version: 0.0.5
 Summary: Linter and formatter for source code files and text files
 Home-page: https://github.com/DavidPal/whitespace-format
 License: MIT
@@ -14,7 +14,7 @@ Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
 Project-URL: Repository, https://github.com/DavidPal/whitespace-format
 Description-Content-Type: text/markdown
 
@@ -22,7 +22,7 @@ Description-Content-Type: text/markdown
 
 [![CircleCI](https://dl.circleci.com/status-badge/img/gh/DavidPal/whitespace-format/tree/main.svg?style=svg)](https://dl.circleci.com/status-badge/redirect/gh/DavidPal/whitespace-format/tree/main)
 
-![GitHub Action](https://github.com/DavidPal/whitespace-format/actions/workflows/build.yaml/badge.svg)
+[![Build, lint and test](https://github.com/DavidPal/whitespace-format/actions/workflows/build.yaml/badge.svg)](https://github.com/DavidPal/whitespace-format/actions/workflows/build.yaml)
 
 Linter and formatter for source code files and text files.
 
@@ -32,7 +32,7 @@ MarkDown, LaTeX) before checking them into a version control system.
 
 The features include:
 
-* Auto-detection of new line markers (Linux `\n`, Windows `\r\r`, Mac `\r`).
+* Auto-detection of new line markers (Linux `\n`, Windows `\r\n`, Mac `\r`).
 * Add a new line marker at the end of the file if it is missing.
 * Make new line markers consistent.
 * Remove empty lines at the end of the file.
@@ -62,17 +62,18 @@ whitespace-format \
     --normalize-new-line-markers \
     foo.txt  my_project/
 ```
-The command above formats `foo.txt` and all files contained `my_project/` and
-its subdirectories. Files that contain `.git/` or `.idea/` in their (relative)
-path are excluded. For example, files in `my_project/.git/` and files in
-`my_project/.idea/` are excluded. Likewise, files ending with `*.pyc` are
-excluded.
-
-If you want only know if any changes **would be** made, add `--check-only` option:
+The command above formats `foo.txt` and all files contained in `my_project/`
+directory and its subdirectories. Files that contain `.git/` or `.idea/` in
+their (relative) path are excluded. For example, files in `my_project/.git/`
+and files in `my_project/.idea/` are excluded. Likewise, files ending with
+`*.pyc` are excluded.
+
+If you want to know only if any changes **would be** made, add `--check-only`
+option:
 ```shell
 whitespace-format \
-    --exclude ".git/|.idea/|.pyc$" \
     --check-only \
+    --exclude ".git/|.idea/|.pyc$" \
     --new-line-marker linux \
     --normalize-new-line-markers \
     foo.txt  my_project/
@@ -97,7 +98,7 @@ The regular expression is evaluated on the path of each file.
 This option is ignored when `--add-new-line-marker-at-end-of-file` is used.
 Empty lines at the end of the file are removed.
 * `--normalize-new-line-markers` -- Make new line markers consistent in each file
-by replacing `\\r\\n`, `\\n`, and `\r` with a consistent new line marker.
+by replacing `\r\n`, `\n`, and `\r` with a consistent new line marker.
 * `--remove-trailing-whitespace` -- Remove whitespace at the end of each line.
 * `--remove-trailing-empty-lines` -- Remove empty lines at the end of each file.
 * `--new-line-marker=MARKER` -- This option specifies what new line marker to use.
@@ -105,9 +106,9 @@ by replacing `\\r\\n`, `\\n`, and `\r` with a consistent new line marker.
   * `auto` -- Use new line marker that is the most common in each individual file.
   If no new line marker is present in the file, Linux `\n` is used.
   This is the default option.
-  * `linux` -- Use Linux new line marker `\\n`.
-  * `mac` -- Use Mac new line marker `\\r`.
-  * `windows` -- Use Windows new line marker `\\r\\n`.
+  * `linux` -- Use Linux new line marker `\n`.
+  * `mac` -- Use Mac new line marker `\r`.
+  * `windows` -- Use Windows new line marker `\r\n`.
 * `--encoding` -- Text encoding for both reading and writing files. Default encoding is `utf-8`.
 List of supported encodings can be found at
 https://docs.python.org/3/library/codecs.html#standard-encodings
@@ -162,7 +163,7 @@ Default value is `-1`.
 
 * `--normalize-non-standard-whitespace=MODE` -- Replace or remove
 non-standard whitespace characters (`\v` and `\f`). `MODE` must be one of the following:
-  * `ignore` -- Leave `\v` and `f` as is. This is the default option.
+  * `ignore` -- Leave `\v` and `\f` as is. This is the default option.
   * `replace` -- Replace any occurrence of `\v` or `\f` with a single space.
   * `remove` -- Remove all occurrences of `\v` and `\f`
 
@@ -182,13 +183,7 @@ MIT
     brew install poetry
     ```
 
-3) Create Python virtual environment with the correct Python version:
-   ```shell
-   make install-python
-   make create-environment
-   ```
-
-4) Add the following lines to `.zshrc` or `.bash_profile` and restart the terminal:
+3) Add the following lines to `.zshrc` or `.bash_profile` and restart the terminal:
    ```shell
    # Pyenv settings
    export PYENV_ROOT="$HOME/.pyenv"
@@ -197,6 +192,12 @@ MIT
    eval "$(pyenv virtualenv-init -)"
    ```
 
+4) Create Python virtual environment with the correct Python version:
+   ```shell
+   make install-python
+   make create-environment
+   ```
+
 5) Install all dependencies
     ```shell
     make install-dependecies

{whitespace_format-0.0.3 → whitespace_format-0.0.5}/README.md

@@ -2,7 +2,7 @@
 
 [![CircleCI](https://dl.circleci.com/status-badge/img/gh/DavidPal/whitespace-format/tree/main.svg?style=svg)](https://dl.circleci.com/status-badge/redirect/gh/DavidPal/whitespace-format/tree/main)
 
-![GitHub Action](https://github.com/DavidPal/whitespace-format/actions/workflows/build.yaml/badge.svg)
+[![Build, lint and test](https://github.com/DavidPal/whitespace-format/actions/workflows/build.yaml/badge.svg)](https://github.com/DavidPal/whitespace-format/actions/workflows/build.yaml)
 
 Linter and formatter for source code files and text files.
 
@@ -12,7 +12,7 @@ MarkDown, LaTeX) before checking them into a version control system.
 
 The features include:
 
-* Auto-detection of new line markers (Linux `\n`, Windows `\r\r`, Mac `\r`).
+* Auto-detection of new line markers (Linux `\n`, Windows `\r\n`, Mac `\r`).
 * Add a new line marker at the end of the file if it is missing.
 * Make new line markers consistent.
 * Remove empty lines at the end of the file.
@@ -42,17 +42,18 @@ whitespace-format \
     --normalize-new-line-markers \
     foo.txt  my_project/
 ```
-The command above formats `foo.txt` and all files contained `my_project/` and
-its subdirectories. Files that contain `.git/` or `.idea/` in their (relative)
-path are excluded. For example, files in `my_project/.git/` and files in
-`my_project/.idea/` are excluded. Likewise, files ending with `*.pyc` are
-excluded.
-
-If you want only know if any changes **would be** made, add `--check-only` option:
+The command above formats `foo.txt` and all files contained in `my_project/`
+directory and its subdirectories. Files that contain `.git/` or `.idea/` in
+their (relative) path are excluded. For example, files in `my_project/.git/`
+and files in `my_project/.idea/` are excluded. Likewise, files ending with
+`*.pyc` are excluded.
+
+If you want to know only if any changes **would be** made, add `--check-only`
+option:
 ```shell
 whitespace-format \
-    --exclude ".git/|.idea/|.pyc$" \
     --check-only \
+    --exclude ".git/|.idea/|.pyc$" \
     --new-line-marker linux \
     --normalize-new-line-markers \
     foo.txt  my_project/
@@ -77,7 +78,7 @@ The regular expression is evaluated on the path of each file.
 This option is ignored when `--add-new-line-marker-at-end-of-file` is used.
 Empty lines at the end of the file are removed.
 * `--normalize-new-line-markers` -- Make new line markers consistent in each file
-by replacing `\\r\\n`, `\\n`, and `\r` with a consistent new line marker.
+by replacing `\r\n`, `\n`, and `\r` with a consistent new line marker.
 * `--remove-trailing-whitespace` -- Remove whitespace at the end of each line.
 * `--remove-trailing-empty-lines` -- Remove empty lines at the end of each file.
 * `--new-line-marker=MARKER` -- This option specifies what new line marker to use.
@@ -85,9 +86,9 @@ by replacing `\\r\\n`, `\\n`, and `\r` with a consistent new line marker.
   * `auto` -- Use new line marker that is the most common in each individual file.
   If no new line marker is present in the file, Linux `\n` is used.
   This is the default option.
-  * `linux` -- Use Linux new line marker `\\n`.
-  * `mac` -- Use Mac new line marker `\\r`.
-  * `windows` -- Use Windows new line marker `\\r\\n`.
+  * `linux` -- Use Linux new line marker `\n`.
+  * `mac` -- Use Mac new line marker `\r`.
+  * `windows` -- Use Windows new line marker `\r\n`.
 * `--encoding` -- Text encoding for both reading and writing files. Default encoding is `utf-8`.
 List of supported encodings can be found at
 https://docs.python.org/3/library/codecs.html#standard-encodings
@@ -142,7 +143,7 @@ Default value is `-1`.
 
 * `--normalize-non-standard-whitespace=MODE` -- Replace or remove
 non-standard whitespace characters (`\v` and `\f`). `MODE` must be one of the following:
-  * `ignore` -- Leave `\v` and `f` as is. This is the default option.
+  * `ignore` -- Leave `\v` and `\f` as is. This is the default option.
   * `replace` -- Replace any occurrence of `\v` or `\f` with a single space.
   * `remove` -- Remove all occurrences of `\v` and `\f`
 
@@ -162,13 +163,7 @@ MIT
     brew install poetry
     ```
 
-3) Create Python virtual environment with the correct Python version:
-   ```shell
-   make install-python
-   make create-environment
-   ```
-
-4) Add the following lines to `.zshrc` or `.bash_profile` and restart the terminal:
+3) Add the following lines to `.zshrc` or `.bash_profile` and restart the terminal:
    ```shell
    # Pyenv settings
    export PYENV_ROOT="$HOME/.pyenv"
@@ -177,6 +172,12 @@ MIT
    eval "$(pyenv virtualenv-init -)"
    ```
 
+4) Create Python virtual environment with the correct Python version:
+   ```shell
+   make install-python
+   make create-environment
+   ```
+
 5) Install all dependencies
     ```shell
     make install-dependecies

{whitespace_format-0.0.3 → whitespace_format-0.0.5}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "whitespace-format"
-version = "0.0.3"
+version = "0.0.5"
 description = "Linter and formatter for source code files and text files"
 license = "MIT"
 authors = ["David Pal <davidko.pal@gmail.com>"]
@@ -25,10 +25,10 @@ coverage = "^7.2.4"
 flake8 = "^5.0.4"
 flake8-absolute-import = "^1.0.0.1"
 isort = {extras = ["colors"], version = "^5.11.5"}
-mypy = "^1.2.0"
+mypy = "^1.4.1"
 pydocstyle = {extras = ["toml"], version = "^6.3.0"}
-pytest = "^7.3.1"
-pylint = "^2.16.3"
+pytest = "^7.4.1"
+pylint = "^2.17.5"
 pylint-quotes = "^0.2.3"
 
 [build-system]

{whitespace_format-0.0.3 → whitespace_format-0.0.5}/whitespace_format.py

@@ -10,22 +10,26 @@ Usage:
    python whitespace_format.py [OPTIONS] [FILES ...]
 """
 
+from __future__ import annotations
+
 import argparse
+import copy
 import dataclasses
 import pathlib
 import re
 import sys
 from typing import Callable
+from typing import Dict
 from typing import List
 
-VERSION = "0.0.3"
+VERSION = "0.0.5"
 
 # Regular expression that does NOT match any string.
 UNMATCHABLE_REGEX = "$."
 
-NEW_LINE_MARKERS = {
-    "linux": "\n",
+END_OF_LINE_MARKERS = {
     "windows": "\r\n",
+    "linux": "\n",
     "mac": "\r",
 }
 
@@ -68,6 +72,14 @@ def color_print(message: str, parsed_arguments: argparse.Namespace):
     print(message)
 
 
+def string_to_hex(text: str) -> str:
+    """Converts a string into a human-readable hexadecimal representation.
+
+    This function is for debugging purposes only. It is used only during development.
+    """
+    return ":".join(f"{ord(character):02x}" for character in text)
+
+
 def die(error_code: int, message: str = ""):
     """Exits the script."""
     if message:
@@ -78,7 +90,7 @@ def die(error_code: int, message: str = ""):
 def read_file_content(file_name: str, encoding: str) -> str:
     """Reads content of a file."""
     try:
-        with open(file_name, "r", encoding=encoding) as file:
+        with open(file_name, "r", encoding=encoding, newline="") as file:
             return file.read()
     except IOError as exception:
         die(2, f"Cannot read file '{file_name}': {exception}")
@@ -96,99 +108,192 @@ def write_file(file_name: str, file_content: str, encoding: str):
         die(4, f"Cannot write to file '{file_name}': {exception}")
 
 
-def guess_new_line_marker(file_content: str) -> str:
-    """Guesses newline character for a file.
+@dataclasses.dataclass
+class Line:
+    """Line of a text file.
 
-    The guess is based on the counts of "\n", "\r" and "\rn" in the file.
+    The line is split into two parts:
+        1) Content
+        2) End of line marker ("\n", or "\r", or "\r\n")
     """
-    windows_count = file_content.count("\r\n")
-    linux_count = file_content.count("\n") - windows_count
-    mac_count = file_content.count("\r") - windows_count
-
-    # Pick the new line marker with the highest count.
-    # Break ties according to the ordering: Linux > Windows > Mac.
-    _, _, new_line_marker_guess = max(
-        (linux_count, 3, "\n"),
-        (windows_count, 2, "\r\n"),
-        (mac_count, 1, "\r"),
-    )
 
-    return new_line_marker_guess
+    content: str
+    end_of_line_marker: str
 
+    @staticmethod
+    def create_from_string(line: str) -> Line:
+        """Creates a line from a string.
 
-def remove_trailing_empty_lines(file_content: str) -> str:
-    """Removes trailing empty lines."""
-    reduced_file_content = file_content.rstrip("\r\n")
-    i = len(reduced_file_content)
-    if i >= len(file_content) - 1:
-        return file_content
-    if file_content[i : i + 2] == "\r\n":
-        return file_content[: i + 2]
-    return file_content[: i + 1]
+        The function splits the input into content and end_of_line_marker.
+        """
+        for end_of_line_marker in ["\r\n", "\n", "\r"]:
+            if line.endswith(end_of_line_marker):
+                return Line(line[: -len(end_of_line_marker)], end_of_line_marker)
+        return Line(line, "")
 
+    def to_hex(self):
+        """Returns a human-readable hexadecimal representation of the line.
 
-def remove_trailing_whitespace(file_content: str) -> str:
-    """Removes trailing whitespace from every line."""
-    file_content = re.sub(r"[ \t\f\v]*\n", "\n", file_content)
-    file_content = re.sub(r"[ \t\f\v]*\r", "\r", file_content)
-    file_content = re.sub(r"[ \t\f\v]*$", "", file_content)
-    return file_content
+        This function is for debugging purposes only. It is used only during development.
+        """
+        return f"({string_to_hex(self.content)}, {string_to_hex(self.end_of_line_marker)})"
+
+
+def split_lines(text: str) -> List[Line]:
+    """Splits a string into lines."""
+    lines: List[Line] = []
+    current_line = ""
+    for i, char in enumerate(text):
+        current_line += char
+        if (char == "\n") or (
+            (char == "\r") and ((i >= len(text) - 1) or (not text[i + 1] == "\n"))
+        ):
+            lines.append(Line.create_from_string(current_line))
+            current_line = ""
+
+    if current_line:
+        lines.append(Line.create_from_string(current_line))
 
+    return lines
+
+
+def concatenate_lines(lines: List[Line]) -> str:
+    """Concatenates a list of lines into a single string including end-of-line markers."""
+    return "".join(line.content + line.end_of_line_marker for line in lines)
+
+
+def guess_end_of_line_marker(lines: List[Line]) -> str:
+    """Guesses the end of line marker.
+
+    The function returns the most common end-of-line marker.
+    Ties are broken in order Linux "\n", Mac "\r", Windows "\r\n".
+    If no end-of-line marker is present, default to the Linux "\n" end-of-line marker.
+    """
+    counts: Dict[str, int] = {"\n": 0, "\r": 0, "\r\n": 0}
+    for line in lines:
+        if line.end_of_line_marker in counts:
+            counts[line.end_of_line_marker] += 1
+    max_count = max(counts.values())
+    for end_of_line_marker, count in counts.items():
+        if count == max_count:
+            return end_of_line_marker
+    return "\n"  # This return statement is never executed.
+
+
+def remove_trailing_empty_lines(lines: List[Line]) -> List[Line]:
+    """Removes trailing empty lines.
+
+    If there are no lines, empty list is returned.
+    If all lines are empty, the first line is kept.
+    """
+    num_empty_trailing_lines = 0
+    while (num_empty_trailing_lines < len(lines) - 1) and (
+        not lines[-num_empty_trailing_lines - 1].content
+    ):
+        num_empty_trailing_lines += 1
+    return copy.deepcopy(lines[: len(lines) - num_empty_trailing_lines])
+
+
+def remove_dummy_lines(lines: List[Line]) -> List[Line]:
+    """Remove empty lines that also have empty end-of-line markers."""
+    return [line for line in lines if line.content or line.end_of_line_marker]
+
+
+def remove_trailing_whitespace(lines: List[Line]) -> List[Line]:
+    """Removes trailing whitespace from every line."""
+    lines = [
+        Line(
+            re.sub(r"[ \n\r\t\f\v]*$", "", line.content),
+            line.end_of_line_marker,
+        )
+        for line in lines
+    ]
+    return remove_dummy_lines(lines)
 
-def remove_all_new_line_marker_from_end_of_file(file_content: str) -> str:
-    """Removes all new line markers from the end of the file."""
-    return file_content.rstrip("\n\r")
 
+def normalize_end_of_line_markers(lines: List[Line], new_end_of_line_marker: str) -> List[Line]:
+    """Replaces end-of-line marker in all lines with a new end-of-line marker.
 
-def remove_new_line_marker_from_end_of_file(file_content: str) -> str:
-    """Removes the last new line marker from the last line."""
-    if file_content.endswith("\r\n"):
-        return file_content[:-2]
-    if file_content.endswith("\n") or file_content.endswith("\r"):
-        return file_content[:-1]
-    return file_content
+    Lines without end-of-line markers (i.e. possibly the last line) are left unchanged.
+    """
+    return [
+        Line(line.content, new_end_of_line_marker) if line.end_of_line_marker else line
+        for line in lines
+    ]
 
 
-def add_new_line_marker_at_end_of_file(file_content: str, new_line_marker: str) -> str:
-    """Adds new line marker to the end of file if it is missing."""
-    return remove_new_line_marker_from_end_of_file(file_content) + new_line_marker
+def remove_all_end_of_line_markers_from_end_of_file(lines: List[Line]) -> List[Line]:
+    """Removes all end-of-line markers from the end of the file."""
+    lines = remove_trailing_empty_lines(lines)
+    if not lines:
+        return []
+    lines[-1] = Line(lines[-1].content, "")
+    return remove_dummy_lines(lines)
 
 
-def normalize_new_line_markers(file_content: str, new_line_marker: str) -> str:
-    """Fixes line endings."""
-    file_content = file_content.replace("\r\n", "\n")
-    file_content = file_content.replace("\r", "\n")
-    return file_content.replace("\n", new_line_marker)
+def add_end_of_line_marker_at_end_of_file(
+    lines: List[Line], new_end_of_line_marker: str
+) -> List[Line]:
+    """Adds new end-of-line marker to the end of file if it is missing."""
+    if not lines:
+        return [Line("", new_end_of_line_marker)]
+    lines = copy.deepcopy(lines)
+    lines[-1] = Line(lines[-1].content, new_end_of_line_marker)
+    return lines
 
 
-def normalize_empty_file(file_content: str, mode: str, new_line_marker: str) -> str:
+def normalize_empty_file(lines: List[Line], mode: str, new_end_of_line_marker: str) -> List[Line]:
     """Replaces file with an empty file."""
     if mode == "empty":
-        return ""
+        return []
     if mode == "one-line":
-        return new_line_marker
-    return file_content
+        return [Line("", new_end_of_line_marker)]
+    return copy.deepcopy(lines)
 
 
-def is_whitespace_only(file_content: str) -> bool:
+def is_whitespace_only(lines: List[Line]) -> bool:
     """Determines if file consists only of whitespace."""
-    return not file_content.strip(" \n\r\t\v\f")
+    for line in lines:
+        if line.content.strip(" \n\r\t\v\f"):
+            return False
+    return True
 
 
-def normalize_non_standard_whitespace(file_content: str, mode: str) -> str:
+def normalize_non_standard_whitespace(lines: List[Line], mode: str) -> List[Line]:
     """Removes non-standard whitespace characters."""
     if mode == "ignore":
-        return file_content
+        return copy.deepcopy(lines)
     if mode == "replace":
-        return file_content.translate(str.maketrans("\v\f", "  ", ""))
-    return file_content.translate(str.maketrans("", "", "\v\f"))
+        return [
+            Line(line.content.translate(str.maketrans("\v\f", "  ", "")), line.end_of_line_marker)
+            for line in lines
+        ]
+    return [
+        Line(line.content.translate(str.maketrans("", "", "\v\f")), line.end_of_line_marker)
+        for line in lines
+    ]
 
 
-def replace_tabs_with_spaces(file_content: str, num_spaces: int) -> str:
+def replace_tabs_with_spaces(lines: List[Line], num_spaces: int) -> List[Line]:
     """Replaces tabs with spaces."""
     if num_spaces < 0:
-        return file_content
-    return file_content.replace("\t", num_spaces * " ")
+        return copy.deepcopy(lines)
+    return [
+        Line(line.content.replace("\t", num_spaces * " "), line.end_of_line_marker)
+        for line in lines
+    ]
+
+
+def compute_difference(original_lines: List[Line], new_lines: List[Line]) -> List[int]:
+    """Computes the indices of lines that differ."""
+    line_numbers = [
+        line_number
+        for line_number, (original_line, new_line) in enumerate(zip(original_lines, new_lines))
+        if not original_line == new_line
+    ]
+    if len(original_lines) != len(new_lines):
+        line_numbers.append(min(len(original_lines), len(new_lines)))
+    return line_numbers
 
 
 @dataclasses.dataclass
@@ -199,25 +304,36 @@ class ChangeDescription:
     change: str
 
 
+@dataclasses.dataclass
+class LineChange:
+    """Description of a change on a particular line."""
+
+    check_only: str
+    change: str
+    line_number: int
+
+
 class FileContentTracker:
     """Tracks changes of the content of a file as it undergoes formatting."""
 
-    def __init__(self, file_content: str):
+    def __init__(self, lines: List[Line]):
         """Initializes an instance of the file content tracker."""
-        self.initial_content = file_content
-        self.file_content = file_content
-        self.changes: List[ChangeDescription] = []
+        self.initial_lines = lines
+        self.lines = copy.deepcopy(lines)
+        self.line_changes: List[LineChange] = []
 
-    def format(self, change: ChangeDescription, function: Callable[..., str], *args):
+    def format(self, change: ChangeDescription, function: Callable[..., List[Line]], *args):
         """Applies a change to the content of the file."""
-        previous_content = self.file_content
-        self.file_content = function(self.file_content, *args)
-        if previous_content != self.file_content:
-            self.changes.append(change)
+        previous_content = self.lines
+        self.lines = function(self.lines, *args)
+        if previous_content != self.lines:
+            line_numbers = compute_difference(previous_content, self.lines)
+            for line_number in line_numbers:
+                self.line_changes.append(LineChange(change.check_only, change.change, line_number))
 
     def is_changed(self) -> bool:
         """Determines if the file content has changed."""
-        return self.file_content != self.initial_content
+        return self.lines != self.initial_lines
 
 
 def format_file_content(
@@ -225,12 +341,12 @@ def format_file_content(
     parsed_arguments: argparse.Namespace,
 ):
     """Formats the content of file represented as a string."""
-    new_line_marker = NEW_LINE_MARKERS.get(
+    new_line_marker = END_OF_LINE_MARKERS.get(
         parsed_arguments.new_line_marker,
-        guess_new_line_marker(file_content_tracker.initial_content),
+        guess_end_of_line_marker(file_content_tracker.initial_lines),
     )
 
-    if is_whitespace_only(file_content_tracker.initial_content):
+    if is_whitespace_only(file_content_tracker.initial_lines):
         changes = {
             "ignore": ChangeDescription("", ""),
             "empty": ChangeDescription(
@@ -246,7 +362,7 @@ def format_file_content(
                 ),
             ),
         }
-        if not file_content_tracker.initial_content:
+        if not file_content_tracker.initial_lines:
             file_content_tracker.format(
                 changes[parsed_arguments.normalize_empty_files],
                 normalize_empty_file,
@@ -265,8 +381,8 @@ def format_file_content(
         if parsed_arguments.remove_trailing_whitespace:
             file_content_tracker.format(
                 ChangeDescription(
-                    check_only="Whitespace at the end of line(s) needs to be removed.",
-                    change="Whitespace at the end of line(s) was removed.",
+                    check_only="Whitespace at the end of line needs to be removed.",
+                    change="Whitespace at the end of line was removed.",
                 ),
                 remove_trailing_whitespace,
             )
@@ -308,7 +424,7 @@ def format_file_content(
                     ),
                     change=f"New line marker(s) were replaced with {repr(new_line_marker)}.",
                 ),
-                normalize_new_line_markers,
+                normalize_end_of_line_markers,
                 new_line_marker,
             )
 
@@ -320,7 +436,7 @@ def format_file_content(
                     change=f"New line marker was added to the end of the file, "
                     f"or replaced with {repr(new_line_marker)}.",
                 ),
-                add_new_line_marker_at_end_of_file,
+                add_end_of_line_marker_at_end_of_file,
                 new_line_marker,
             )
         elif parsed_arguments.remove_new_line_marker_from_end_of_file:
@@ -329,14 +445,15 @@ def format_file_content(
                     check_only="New line marker(s) need to removed from the end of the file.",
                     change="New line marker(s) were removed from the end of the file.",
                 ),
-                remove_all_new_line_marker_from_end_of_file,
+                remove_all_end_of_line_markers_from_end_of_file,
             )
 
 
 def reformat_file(file_name: str, parsed_arguments: argparse.Namespace) -> bool:
     """Reformats a file."""
     file_content = read_file_content(file_name, parsed_arguments.encoding)
-    file_content_tracker = FileContentTracker(file_content)
+    lines = split_lines(file_content)
+    file_content_tracker = FileContentTracker(lines)
     format_file_content(file_content_tracker, parsed_arguments)
     is_changed = file_content_tracker.is_changed()
     if parsed_arguments.verbose:
@@ -348,9 +465,11 @@ def reformat_file(file_name: str, parsed_arguments: argparse.Namespace) -> bool:
                 f"[RED]needs to be formatted[RESET_ALL]",
                 parsed_arguments,
             )
-            for change in file_content_tracker.changes:
+            for line_change in file_content_tracker.line_changes:
                 color_print(
-                    f"   [BOLD][BLUE]↳ [WHITE]{change.check_only}[RESET_ALL]", parsed_arguments
+                    f"   [BOLD][BLUE]↳ line {line_change.line_number + 1}: "
+                    f"[WHITE]{line_change.check_only}[RESET_ALL]",
+                    parsed_arguments,
                 )
         else:
             if parsed_arguments.verbose:
@@ -362,9 +481,15 @@ def reformat_file(file_name: str, parsed_arguments: argparse.Namespace) -> bool:
     else:
         if is_changed:
             color_print(f"[WHITE]Reformatted [BOLD]{file_name}[RESET_ALL]", parsed_arguments)
-            for change in file_content_tracker.changes:
-                color_print(f"   [BOLD][BLUE]↳ [WHITE]{change.change}[RESET_ALL]", parsed_arguments)
-            write_file(file_name, file_content_tracker.file_content, parsed_arguments.encoding)
+            for line_change in file_content_tracker.line_changes:
+                color_print(
+                    f"   [BOLD][BLUE]↳ line {line_change.line_number + 1}: "
+                    f"[WHITE]{line_change.change}[RESET_ALL]",
+                    parsed_arguments,
+                )
+            write_file(
+                file_name, concatenate_lines(file_content_tracker.lines), parsed_arguments.encoding
+            )
         else:
             if parsed_arguments.verbose:
                 color_print(f"[WHITE]{file_name} [BLUE]left unchanged[RESET_ALL]", parsed_arguments)

whitespace_format-0.0.3/setup.py

@@ -1,25 +0,0 @@
-# -*- coding: utf-8 -*-
-from setuptools import setup
-
-modules = \
-['whitespace_format']
-entry_points = \
-{'console_scripts': ['whitespace-format = whitespace_format:main']}
-
-setup_kwargs = {
-    'name': 'whitespace-format',
-    'version': '0.0.3',
-    'description': 'Linter and formatter for source code files and text files',
-    'long_description': '# whitespace-format\n\n[![CircleCI](https://dl.circleci.com/status-badge/img/gh/DavidPal/whitespace-format/tree/main.svg?style=svg)](https://dl.circleci.com/status-badge/redirect/gh/DavidPal/whitespace-format/tree/main)\n\n![GitHub Action](https://github.com/DavidPal/whitespace-format/actions/workflows/build.yaml/badge.svg)\n\nLinter and formatter for source code files and text files.\n\nThe purpose of this tool is to normalize source code files (e.g. Python, Java,\nC/C++, Ruby, Go, JavaScript, etc.) and text files (HTML, JSON, YAML, CSV,\nMarkDown, LaTeX) before checking them into a version control system.\n\nThe features include:\n\n* Auto-detection of new line markers (Linux `\\n`, Windows `\\r\\r`, Mac `\\r`).\n* Add a new line marker at the end of the file if it is missing.\n* Make new line markers consistent.\n* Remove empty lines at the end of the file.\n* Remove whitespace at the end of each line.\n* Replace tabs with spaces.\n* Remove/replace non-standard whitespace characters.\n\nThe formatting changes are\n[idempotent](https://en.wikipedia.org/wiki/Idempotence), i.e., running the tool\nsecond time (with the same parameters) has no effect.\n\n## Installation\n\n```shell\npip install whitespace-format\n```\n\nInstallation requires Python 3.7.5 or higher.\n\n## Usage\n\nA sample command that formats source code files:\n```shell\nwhitespace-format \\\n    --exclude ".git/|.idea/|.pyc$" \\\n    --new-line-marker linux \\\n    --normalize-new-line-markers \\\n    foo.txt  my_project/\n```\nThe command above formats `foo.txt` and all files contained `my_project/` and\nits subdirectories. Files that contain `.git/` or `.idea/` in their (relative)\npath are excluded. For example, files in `my_project/.git/` and files in\n`my_project/.idea/` are excluded. Likewise, files ending with `*.pyc` are\nexcluded.\n\nIf you want only know if any changes **would be** made, add `--check-only` option:\n```shell\nwhitespace-format \\\n    --exclude ".git/|.idea/|.pyc$" \\\n    --check-only \\\n    --new-line-marker linux \\\n    --normalize-new-line-markers \\\n    foo.txt  my_project/\n```\nThis command can be used as a validation step before checking the source files\ninto a version control system. The command outputs non-zero exit code if any\nof the files would be formatted.\n\n### Options\n\n* `--check-only` -- Do not format files. Only report which files would be formatted.\n* `--follow-symlinks` -- Follow symbolic links when searching for files.\n* `--exclude=REGEX` -- Regular expression that specifies which files to exclude.\nThe regular expression is evaluated on the path of each file.\n* `--verbose` -- Print more messages than normally.\n* `--quiet` -- Do not print any messages, except for errors when reading or writing files.\n\n### Formatting options\n\n* `--add-new-line-marker-at-end-of-file` -- Add missing new line marker at end of each file.\n* `--remove-new-line-marker-from-end-of-file` -- Remove all new line marker(s) from the end of each file.\nThis option is ignored when `--add-new-line-marker-at-end-of-file` is used.\nEmpty lines at the end of the file are removed.\n* `--normalize-new-line-markers` -- Make new line markers consistent in each file\nby replacing `\\\\r\\\\n`, `\\\\n`, and `\\r` with a consistent new line marker.\n* `--remove-trailing-whitespace` -- Remove whitespace at the end of each line.\n* `--remove-trailing-empty-lines` -- Remove empty lines at the end of each file.\n* `--new-line-marker=MARKER` -- This option specifies what new line marker to use.\n`MARKER` must be one of the following:\n  * `auto` -- Use new line marker that is the most common in each individual file.\n  If no new line marker is present in the file, Linux `\\n` is used.\n  This is the default option.\n  * `linux` -- Use Linux new line marker `\\\\n`.\n  * `mac` -- Use Mac new line marker `\\\\r`.\n  * `windows` -- Use Windows new line marker `\\\\r\\\\n`.\n* `--encoding` -- Text encoding for both reading and writing files. Default encoding is `utf-8`.\nList of supported encodings can be found at\nhttps://docs.python.org/3/library/codecs.html#standard-encodings\n\nNote that input files can contain an arbitrary mix of new line markers `\\n`,\n`\\r`, `\\r\\n` even within the same file. The option `--new-line-marker`\nspecifies the character that should be in the formatted file.\n\nAn opinionated combination of options is:\n```shell\nwhitespace-format \\\n    --new-line-marker=linux \\\n    --add-new-line-marker-at-end-of-file \\\n    --normalize-new-line-markers \\\n    --remove-trailing-whitespace \\\n    --remove-trailing-empty-lines \\\n    foo.txt  my_project/\n```\nThis should work well for common programming languages (e.g. Python, Java,\nC/C++, JavaScript) and common text file formats (e.g. CSV, LaTeX, JSON, YAML,\nHTML, MarkDown).\n\n### Empty files\n\nThere are separate options for handling empty files and files consisting of\nwhitespace characters only:\n\n* `--normalize-empty-files=MODE`\n* `--normalize-whitespace-only-files=MODE`\n\nwhere `MODE` is one of the following:\n\n* `ignore` -- Leave the file as is. This is the default option.\n* `empty` -- Replace the file with an empty file.\n* `one-line` -- Replace each file with a file consisting of a single new line marker.\n\nDepending on the mode, an empty file or a whitespace-only file will be either\nignored, replaced by a zero-byte file, or replaced by a file consisting of\nsingle end of line marker.\n\nIf `--normalize-whitespace-only-files` is set to `empty`,\n`--normalize-empty-files setting` set to `empty` as well. In other words,\ncombination `--normalize-whitespace-only-files=empty` and\n`--normalize-empty-files=one-line` is not allowed, since it would lead to\nbehavior that is not idempotent.\n\n### Special characters\n\n* `--replace-tabs-with-spaces=N` -- Replace tabs with spaces.\nWhere is `N` is the number of spaces. If `N` is negative, tabs are not replaced.\nDefault value is `-1`.\n\n* `--normalize-non-standard-whitespace=MODE` -- Replace or remove\nnon-standard whitespace characters (`\\v` and `\\f`). `MODE` must be one of the following:\n  * `ignore` -- Leave `\\v` and `f` as is. This is the default option.\n  * `replace` -- Replace any occurrence of `\\v` or `\\f` with a single space.\n  * `remove` -- Remove all occurrences of `\\v` and `\\f`\n\n## License\n\nMIT\n\n## MacOS development setup\n\n1) Make sure you have [brew](https://brew.sh/) package manager installed.\n\n2) Install [pyenv](https://github.com/pyenv/pyenv), [pyenv-virtualenv](https://github.com/pyenv/pyenv-virtualenv)\n   and [poetry](https://python-poetry.org/):\n    ```shell\n    brew install pyenv\n    brew install pyenv-virtualenv\n    brew install poetry\n    ```\n\n3) Create Python virtual environment with the correct Python version:\n   ```shell\n   make install-python\n   make create-environment\n   ```\n\n4) Add the following lines to `.zshrc` or `.bash_profile` and restart the terminal:\n   ```shell\n   # Pyenv settings\n   export PYENV_ROOT="$HOME/.pyenv"\n   export PATH="$PYENV_ROOT/bin:$PATH"\n   eval "$(pyenv init --path)"\n   eval "$(pyenv virtualenv-init -)"\n   ```\n\n5) Install all dependencies\n    ```shell\n    make install-dependecies\n    ```\n\nIf you need to delete the Python virtual environment, you can do so with the\ncommand `make delete-environment`.\n\n## Running unit tests and code checks\n\nIf you make code change, run unit tests and code checks with the command:\n```shell\nmake clean whitespace-format-check isort-check black-check flake8 pydocstyle pylint mypy test coverage\n```\n\nEach make target runs different checks:\n- `clean` deletes temporary files\n- `whitespace-format-check` runs [whitespace-format](https://github.com/DavidPal/whitespace-format) checker on all files\n- `isort-check` runs [isort](https://pycqa.github.io/isort/) checker of imports in `*.py` files\n- `black-check` runs [black](https://github.com/psf/black/) code format checker on `*.py` files\n- `flake8` runs [flake8](https://flake8.pycqa.org/) code style checker on `*.py` files\n- `pydocstyle` runs [pydocstyle](http://www.pydocstyle.org/) docstring checker on `*.py` files\n- `pylint` runs [pylint](https://pylint.org/) code checker on `*.py` files\n- `mypy` runs [mypy](http://mypy-lang.org/) type checker on `*.py` files\n- `test` runs unit tests\n- `coverage` generates code coverage report\n\nYou can automatically format code with the command:\n```shell\nmake isort-format black-format whitespace-format\n```\n\n## Modifying dependencies\n\nThe list of Python packages that this project depends on is specified in\n`pyproject.toml` and in `poetry.lock` files. The file `pyproject.toml` can be\nedited by humans. The file `poetry.lock` is automatically generated by\n`poetry`.\n\nInstall a development dependency with the command:\n```shell\npoetry add --dev <some_new_python_tool>\n```\n\nInstall a new production dependency with the command:\n```shell\npoetry add <some_python_library>\n```\n\n### Manual modification of `pyproject.toml`\n\nInstead of using `poetry add` command, you can edit `pyproject.toml` file. Then,\nregenerate `poetry.lock` file with the command:\n```shell\npoetry lock\n```\nor the command:\n```shell\npoetry lock --no-update\n```\nThe latter command does not update already locked packages.\n\n### Fixing broken Python environment\n\nIf your Python virtual environment becomes broken or polluted with unnecessary\npackages, delete it, recreate it from scratch and install dependencies a fresh\nwith the following commands:\n```shell\nmake delete-environment\nmake create-environment\nmake install-dependencies\n```\n',
-    'author': 'David Pal',
-    'author_email': 'davidko.pal@gmail.com',
-    'maintainer': 'None',
-    'maintainer_email': 'None',
-    'url': 'https://github.com/DavidPal/whitespace-format',
-    'py_modules': modules,
-    'entry_points': entry_points,
-    'python_requires': '>=3.7.2,<4.0.0',
-}
-
-
-setup(**setup_kwargs)