pywaybackup 3.1.0__tar.gz → 3.3.0__tar.gz

{pywaybackup-3.1.0/pywaybackup.egg-info → pywaybackup-3.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.2
+Metadata-Version: 2.4
 Name: pywaybackup
-Version: 3.1.0
+Version: 3.3.0
 Summary: Query and download archive.org as simple as possible.
 Author-email: bitdruid <bitdruid@outlook.com>
 License: MIT License
@@ -29,18 +29,19 @@ Project-URL: homepage, https://github.com/bitdruid/python-wayback-machine-downlo
 Requires-Python: >=3.8
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: pysqlite3-binary==0.5.4
-Requires-Dist: requests==2.31.0
-Requires-Dist: tqdm==4.66.2
+Requires-Dist: pysqlite3-binary==0.5.4; sys_platform == "linux"
+Requires-Dist: pysqlite-binary; sys_platform == "win32"
+Requires-Dist: requests==2.32.3
+Requires-Dist: tqdm==4.67.1
 Requires-Dist: python-magic==0.4.27; sys_platform == "linux"
 Requires-Dist: python-magic-bin==0.4.14; sys_platform == "win32"
+Dynamic: license-file
 
 # python wayback machine downloader
 
 [![PyPI](https://img.shields.io/pypi/v/pywaybackup)](https://pypi.org/project/pywaybackup/)
 [![PyPI - Downloads](https://img.shields.io/pypi/dm/pywaybackup)](https://pypi.org/project/pywaybackup/)
 ![Python Version](https://img.shields.io/badge/Python-3.8-blue)
-<!-- ![Python_Sqlite3 Version](https://img.shields.io/badge/Python_Sqlite3-3.35-blue) -->
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
 Downloading archived web pages from the [Wayback Machine](https://archive.org/web/).
@@ -54,23 +55,27 @@ This tool allows you to download content from the Wayback Machine (archive.org).
 ### Pip
 
 1. Install the package <br>
-   ```pip install pywaybackup```
+   `pip install pywaybackup`
 2. Run the tool <br>
-   ```waybackup -h```
+   `waybackup -h`
 
 ### Manual
 
 1. Clone the repository <br>
-   ```git clone https://github.com/bitdruid/python-wayback-machine-downloader.git```
+   `git clone https://github.com/bitdruid/python-wayback-machine-downloader.git`
 2. Install <br>
-   ```pip install .```
+   `pip install .`
    - in a virtual env or use `--break-system-package`
 
-## Usage infos - important notes
+## notes / issues / hints
 
-- Linux recommended: On Windows machines, the path length is limited. This can only be overcome by editing the registry. Files that exceed the path length will not be downloaded.
-- If you query an explicit file (e.g. a query-string `?query=this` or `login.html`), the `--explicit`-argument is recommended as a wildcard query may lead to an empty result.
+- Linux recommended: On Windows machines, the path length is limited. Files that exceed the path length will not be downloaded.
 - The tool uses a sqlite database to handle snapshots. The database will only persist while the download is running.
+- If you query an explicit file (e.g. a query-string `?query=this` or `login.html`), the `--explicit`-argument is recommended as a wildcard query may lead to an empty result.
+- Downloading directly into a network share is not recommended. The sqlite locking mechanism may cause issues. If you need to download into a network share, set the `--metadata` argument to a local path.
+
+<br>
+<br>
 
 ## Arguments
 
@@ -83,6 +88,7 @@ This tool allows you to download content from the Wayback Machine (archive.org).
   The URL of the web page to download. This argument is required.
 
 #### Mode Selection (Choose One)
+
 - **`-a`**, **`--all`**:<br>
   Download snapshots of all timestamps. You will get a folder per timestamp with the files available at that time.
 - **`-l`**, **`--last`**:<br>
@@ -92,57 +98,77 @@ This tool allows you to download content from the Wayback Machine (archive.org).
 - **`-s`**, **`--save`**:<br>
   Save a page to the Wayback Machine. (beta)
 
-### Optional query parameters
+#### Optional query parameters
 
 - **`-e`**, **`--explicit`**:<br>
   Only download the explicit given URL. No wildcard subdomains or paths. Use e.g. to get root-only snapshots. This is recommended for explicit files like `login.html` or `?query=this`.
 
-- **`--filetype`** `<filetype>`:<br>
-  Specify filetypes to download. Default is all filetypes. Separate multiple filetypes with a comma. Example: `--filetype jpg,css,js`. A filter will result in a filtered cdx-file. So if you want to download all files later, you need to query again without the filter. Filetypes are filtered as they are in the snapshot. So if there is no explicit `html` file in the path (common practice) then you cant filter them.
-
 - **`--limit`** `<count>`:<br>
-Limits the amount of snapshots to query from the CDX server. If an existing CDX file is injected, the limit will have no effect. So you would need to set `--keep`.
+  Limits the amount of snapshots to query from the CDX server. If an existing CDX file is injected, the limit will have no effect. So you would need to set `--keep`.
 
 - **Range Selection:**<br>
   Specify the range in years or a specific timestamp either start, end, or both. If you specify the `range` argument, the `start` and `end` arguments will be ignored. Format for timestamps: YYYYMMDDhhmmss. You can only give a year or increase specificity by going through the timestamp starting on the left.<br>
   (year 2019, year 2019, year+month+day 20190101, year+month+day+hour 2019010112)
-   - **`-r`**, **`--range`**:<br>
-     Specify the range in years for which to search and download snapshots.
-   - **`--start`**:<br>
-     Timestamp to start searching.
-   - **`--end`**:<br>
-     Timestamp to end searching.
 
-### Behavior manipulation
+  - **`-r`**, **`--range`**:<br>
+    Specify the range in years for which to search and download snapshots.
+  - **`--start`**:<br>
+    Timestamp to start searching.
+  - **`--end`**:<br>
+    Timestamp to end searching.
+
+- **Filtering:**<br>
+  A filter will result in a filtered cdx-file. So if you want to download all files later, you need to query again without the filter.
+
+  - **`--filetype`** `<filetype>`:<br>
+    Specify filetypes to download. Default is all filetypes. Separate multiple filetypes with a comma. Example: `--filetype jpg,css,js`. Filetypes are filtered as they are in the snapshot. So if there is no explicit `html` file in the path (common practice) then you cant filter them.
+
+  - **`--statuscode`** `<statuscode>`:<br>
+    Specify HTTP status codes to download. Default is all statuscodes. Separate multiple status codes with a comma. Example: `--statuscode 200,301`. Pywaybackup will try to download any snapshot regardless of it's statuscode. For 404 of course this means logged errors and corresponding entries in the csv. However, you may want to get a csv that includes these negative attempts for your needs.<br>
+    Common status codes you may want to handle/filter:
+      - `200` (OK)
+      - `301` (Moved Permanently - will redirect snapshot)
+      - `404` (Not Found - snapshot seems to be empty)
+      - `500` (Internal Server Error - snapshot is at least for now not available)
+
+### Optional
+
+#### Behavior Manipulation
 
 - **`-o`**, **`--output`**:<br>
-Defaults to `waybackup_snapshots` in the current directory. The folder where downloaded files will be saved.
+  Defaults to `waybackup_snapshots` in the current directory. The folder where downloaded files will be saved.
+
+- **`-m`**, **`--metadata`**<br>
+  Change the folder where metadata will be saved (`cdx`/`db`/`csv`/`log`). Especially if you are downloading into a network share, you SHOULD set this to a local path because sqlite locking mechanism may cause issues with network shares.
+
+- **`--verbose`**:<br>
+  Increase output verbosity.
 
 <!-- - **`--verbosity`** `<level>`:<br>
 Sets verbosity level. Options are `info`and `trace`. Default is `info`. -->
 
 - **`--log`** <!-- `<path>` -->:<br>
-Saves a log file into the output-dir. Named as `waybackup_<sanitized_url>.log`.
+  Saves a log file into the output-dir. Named as `waybackup_<sanitized_url>.log`.
 
 - **`--progress`**:<br>
-Shows a progress bar instead of the default output.
+  Shows a progress bar instead of the default output.
 
 - **`--workers`** `<count>`:<br>
-Sets the number of simultaneous download workers. Default is 1, safe range is about 10. Be cautious as too many workers may lead to refused connections from the Wayback Machine.
+  Sets the number of simultaneous download workers. Default is 1, safe range is about 10. Be cautious as too many workers may lead to refused connections from the Wayback Machine.
 
 - **`--no-redirect`**:<br>
-Disables following redirects of snapshots. Useful for preventing timestamp-folder mismatches caused by Archive.org redirects.
-  
+  Disables following redirects of snapshots. Useful for preventing timestamp-folder mismatches caused by Archive.org redirects.
+
 - **`--retry`** `<attempts>`:<br>
-Specifies number of retry attempts for failed downloads.
+  Specifies number of retry attempts for failed downloads.
 
 - **`--delay`** `<seconds>`:<br>
-Specifies delay between download requests in seconds. Default is no delay (0).
+  Specifies delay between download requests in seconds. Default is no delay (0).
 
 <!-- - **`--convert-links`**:<br>
 If set, all links in the downloaded files will be converted to local links. This is useful for offline browsing. The links are converted to the local path structure. Show output with `--verbosity trace`. -->
 
-### Special:
+#### Job Handling:
 
 - **`--reset`**:  
   If set, the job will be reset, and any existing `cdx`, `db`, `csv` files will be **deleted**. This allows you to start the job from scratch without considering previously downloaded data.
@@ -150,47 +176,60 @@ If set, all links in the downloaded files will be converted to local links. This
 - **`--keep`**:  
   If set, all files will be kept after the job is finished. This includes the `cdx` and `db` file. Without this argument, they will be deleted if the job finished successfully.
 
-# Usage 
+<br>
+<br>
+
+## Usage
 
 ### Handling Interrupted Jobs
-When a job is interrupted (by any reason), `pywaybackup` is designed to resume the job from where it left off. It automatically detects existing job data (based on the URL and <u>**optional query parameters**</u> - including output directory) and resumes the process without requiring manual intervention. Here's how the tool handles different scenarios:
-
-- **Default Behavior:** 
-  - On restarting the same job (same URL, <u>**optional query parameters**</u>, and output directory), the tool will:
-    - Reuse the existing `.cdx` and `.db` files.
-    - Resume downloading snapshots from the last successful point.
-    - Skip previously downloaded files to save time and resources.
-
-- **Manual Reset with `--reset`:** 
-  - This command deletes any existing `.cdx` and `.db` files associated with the job and starts the process from scratch.
-  - Useful if:
-    - The previous data is corrupted.
-    - You want to re-query the snapshots without considering previously downloaded data.
-
-- **Preserving Job Data with `--keep`:** 
-  - Normally, `.cdx` and `.db` files are deleted after the job finishes successfully.
-  - Use `--keep` to retain these files for future use (e.g., re-analysis or extending the query later).
-
-> **Note1:** The resumption process only works if the output directory remains the same as the one used during the initial job.
-> 
-> **Note2:** `--reset` will NOT delete the already downloaded files for now. You have to remove them 'by hand'.
-  
-### Example
-
-1. Start downloading all available snapshots:<br>`waybackup -u https://example.com -a`
-2. Interrupt the process `CTRL+C`<br>
-3. The tool will detect the existing job data and resume downloading from the last completed point:<br>`waybackup -u https://example.com -a`
-> **Important:** `waybackup -u https://example.com -c` -> The tool will NOT resume because a necessary identifier-changed
-4. This deletes any existing .cdx and .db files associated with the job and starts the process from scratch:<br>`waybackup -u https://example.com -a --reset`
-5. This ensures all job-related files are kept for future use, such as re-analysis or extending the query later:<br>`waybackup -u https://example.com -a --keep`
-
-## Output path structure
+
+`pywaybackup` resumes interrupted jobs. The tool automatically continues from where it left off.
+
+- Detects existing `.cdx` and `.db` files in an `output dir` to resume downloading from the last successful point.
+- Compares `URL`, `mode`, and `optional query parameters` to ensure automatic resumption.
+- Skips previously downloaded files to save time.
+  > **Note:** Changing URL, mode selection, query parameters or output prevents automatic resumption.
+
+#### Resetting a Job (`--reset`)
+
+- Deletes `.cdx` and `.db` files and restarts the process from scratch.
+- Does **not** remove already downloaded files.
+- `waybackup -u https://example.com -a --reset`
+
+#### Keeping Job Data (`--keep`)
+
+- Normally, `.cdx` and `.db` files are deleted after a successful job.
+- `--keep` preserves them for future re-analysis or extending the query.
+- `waybackup -u https://example.com -a --keep`
+
+<br>
+<br>
+
+## Examples
+
+1. Download a specific single snapshot of all available files (starting from root):<br>
+   `waybackup -u https://example.com -a --start 20210101000000 --end 20210101000000`
+2. Download a specific single snapshot of all available files (starting from a subdirectory):<br>
+   `waybackup -u https://example.com/subdir1/subdir2/assets/ -a --start 20210101000000 --end 20210101000000`
+3. Download a specific single snapshot of the exact given URL (no subdirs):<br>
+   `waybackup -u https://example.com -a --start 20210101000000 --end 20210101000000 --explicit`
+4. Download all snapshots of all available files in the given range:<br>
+   `waybackup -u https://example.com -a --start 20210101000000 --end 20231122000000`
+
+<br>
+<br>
+
+## Output
+
+### Path Structure
 
 The output path is currently structured as follows by an example for the query:<br>
-`http://example.com/subdir1/subdir2/assets/`:
+`http://example.com/subdir1/subdir2/assets/`
 <br><br>
 For the first and last version (`-f` or `-l`):
-- The requested path will only include all files/folders starting from your query-path.
+
+- Will only include all files/folders starting from your query-path.
+
 ```
 your/path/waybackup_snapshots/
 └── the_root_of_your_query/ (example.com/)
@@ -201,8 +240,11 @@ your/path/waybackup_snapshots/
                 ├── style.css
                 ...
 ```
+
 For all versions (`-a`):
-- Will currently create a folder named as the root of your query. Inside this folder, you will find all timestamps and per timestamp the path you requested.
+
+- Will create a folder named as the root of your query. Inside this folder, you will find all timestamps and per timestamp the path you requested.
+
 ```
 your/path/waybackup_snapshots/
 └── the_root_of_your_query/ (example.com/)
@@ -221,7 +263,7 @@ your/path/waybackup_snapshots/
     ...
 ```
 
-## CSV Output
+### CSV
 
 Each snapshot is stored with the following keys/values. These are either stored in a sqlite database while the download is running or saved into a CSV file after the download is finished.
 
@@ -243,15 +285,33 @@ For download queries:
 ]
 ```
 
+### Log
+
+Verbose:
+
+```
+-----> Worker: 2 - Attempt: [1/1] Snapshot ID: [23/81]
+SUCCESS   -> 200 OK
+          -> URL:  https://web.archive.org/web/20240225193302id_/https://example.com/assets/css/custom-styles.css
+          -> FILE: /home/manjaro/Stuff/python-wayback-machine-downloader/waybackup_snapshots/example.com/20240225193302id_/assets/css/custom-styles.css
+```
+
+Non-verbose:
+
+```
+55/81 - W:2 - SUCCESS - 20240225193302 - https://example.com/assets/css/custom-styles.css
+```
+
 ### Debugging
 
 Exceptions will be written into `waybackup_error.log` (each run overwrites the file).
 
-### Known ToDos
-
-- [ ] currently there is no logic to handle if both a http and https version of a page is available
+<br>
+<br>
 
 ## Contributing
 
 I'm always happy for some feature requests to improve the usability of this tool.
 Feel free to give suggestions and report issues. Project is still far from being perfect.
+
+> Please PR from dev into dev.

{pywaybackup-3.1.0 → pywaybackup-3.3.0}/README.md

@@ -3,7 +3,6 @@
 [![PyPI](https://img.shields.io/pypi/v/pywaybackup)](https://pypi.org/project/pywaybackup/)
 [![PyPI - Downloads](https://img.shields.io/pypi/dm/pywaybackup)](https://pypi.org/project/pywaybackup/)
 ![Python Version](https://img.shields.io/badge/Python-3.8-blue)
-<!-- ![Python_Sqlite3 Version](https://img.shields.io/badge/Python_Sqlite3-3.35-blue) -->
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
 Downloading archived web pages from the [Wayback Machine](https://archive.org/web/).
@@ -17,23 +16,27 @@ This tool allows you to download content from the Wayback Machine (archive.org).
 ### Pip
 
 1. Install the package <br>
-   ```pip install pywaybackup```
+   `pip install pywaybackup`
 2. Run the tool <br>
-   ```waybackup -h```
+   `waybackup -h`
 
 ### Manual
 
 1. Clone the repository <br>
-   ```git clone https://github.com/bitdruid/python-wayback-machine-downloader.git```
+   `git clone https://github.com/bitdruid/python-wayback-machine-downloader.git`
 2. Install <br>
-   ```pip install .```
+   `pip install .`
    - in a virtual env or use `--break-system-package`
 
-## Usage infos - important notes
+## notes / issues / hints
 
-- Linux recommended: On Windows machines, the path length is limited. This can only be overcome by editing the registry. Files that exceed the path length will not be downloaded.
-- If you query an explicit file (e.g. a query-string `?query=this` or `login.html`), the `--explicit`-argument is recommended as a wildcard query may lead to an empty result.
+- Linux recommended: On Windows machines, the path length is limited. Files that exceed the path length will not be downloaded.
 - The tool uses a sqlite database to handle snapshots. The database will only persist while the download is running.
+- If you query an explicit file (e.g. a query-string `?query=this` or `login.html`), the `--explicit`-argument is recommended as a wildcard query may lead to an empty result.
+- Downloading directly into a network share is not recommended. The sqlite locking mechanism may cause issues. If you need to download into a network share, set the `--metadata` argument to a local path.
+
+<br>
+<br>
 
 ## Arguments
 
@@ -46,6 +49,7 @@ This tool allows you to download content from the Wayback Machine (archive.org).
   The URL of the web page to download. This argument is required.
 
 #### Mode Selection (Choose One)
+
 - **`-a`**, **`--all`**:<br>
   Download snapshots of all timestamps. You will get a folder per timestamp with the files available at that time.
 - **`-l`**, **`--last`**:<br>
@@ -55,57 +59,77 @@ This tool allows you to download content from the Wayback Machine (archive.org).
 - **`-s`**, **`--save`**:<br>
   Save a page to the Wayback Machine. (beta)
 
-### Optional query parameters
+#### Optional query parameters
 
 - **`-e`**, **`--explicit`**:<br>
   Only download the explicit given URL. No wildcard subdomains or paths. Use e.g. to get root-only snapshots. This is recommended for explicit files like `login.html` or `?query=this`.
 
-- **`--filetype`** `<filetype>`:<br>
-  Specify filetypes to download. Default is all filetypes. Separate multiple filetypes with a comma. Example: `--filetype jpg,css,js`. A filter will result in a filtered cdx-file. So if you want to download all files later, you need to query again without the filter. Filetypes are filtered as they are in the snapshot. So if there is no explicit `html` file in the path (common practice) then you cant filter them.
-
 - **`--limit`** `<count>`:<br>
-Limits the amount of snapshots to query from the CDX server. If an existing CDX file is injected, the limit will have no effect. So you would need to set `--keep`.
+  Limits the amount of snapshots to query from the CDX server. If an existing CDX file is injected, the limit will have no effect. So you would need to set `--keep`.
 
 - **Range Selection:**<br>
   Specify the range in years or a specific timestamp either start, end, or both. If you specify the `range` argument, the `start` and `end` arguments will be ignored. Format for timestamps: YYYYMMDDhhmmss. You can only give a year or increase specificity by going through the timestamp starting on the left.<br>
   (year 2019, year 2019, year+month+day 20190101, year+month+day+hour 2019010112)
-   - **`-r`**, **`--range`**:<br>
-     Specify the range in years for which to search and download snapshots.
-   - **`--start`**:<br>
-     Timestamp to start searching.
-   - **`--end`**:<br>
-     Timestamp to end searching.
 
-### Behavior manipulation
+  - **`-r`**, **`--range`**:<br>
+    Specify the range in years for which to search and download snapshots.
+  - **`--start`**:<br>
+    Timestamp to start searching.
+  - **`--end`**:<br>
+    Timestamp to end searching.
+
+- **Filtering:**<br>
+  A filter will result in a filtered cdx-file. So if you want to download all files later, you need to query again without the filter.
+
+  - **`--filetype`** `<filetype>`:<br>
+    Specify filetypes to download. Default is all filetypes. Separate multiple filetypes with a comma. Example: `--filetype jpg,css,js`. Filetypes are filtered as they are in the snapshot. So if there is no explicit `html` file in the path (common practice) then you cant filter them.
+
+  - **`--statuscode`** `<statuscode>`:<br>
+    Specify HTTP status codes to download. Default is all statuscodes. Separate multiple status codes with a comma. Example: `--statuscode 200,301`. Pywaybackup will try to download any snapshot regardless of it's statuscode. For 404 of course this means logged errors and corresponding entries in the csv. However, you may want to get a csv that includes these negative attempts for your needs.<br>
+    Common status codes you may want to handle/filter:
+      - `200` (OK)
+      - `301` (Moved Permanently - will redirect snapshot)
+      - `404` (Not Found - snapshot seems to be empty)
+      - `500` (Internal Server Error - snapshot is at least for now not available)
+
+### Optional
+
+#### Behavior Manipulation
 
 - **`-o`**, **`--output`**:<br>
-Defaults to `waybackup_snapshots` in the current directory. The folder where downloaded files will be saved.
+  Defaults to `waybackup_snapshots` in the current directory. The folder where downloaded files will be saved.
+
+- **`-m`**, **`--metadata`**<br>
+  Change the folder where metadata will be saved (`cdx`/`db`/`csv`/`log`). Especially if you are downloading into a network share, you SHOULD set this to a local path because sqlite locking mechanism may cause issues with network shares.
+
+- **`--verbose`**:<br>
+  Increase output verbosity.
 
 <!-- - **`--verbosity`** `<level>`:<br>
 Sets verbosity level. Options are `info`and `trace`. Default is `info`. -->
 
 - **`--log`** <!-- `<path>` -->:<br>
-Saves a log file into the output-dir. Named as `waybackup_<sanitized_url>.log`.
+  Saves a log file into the output-dir. Named as `waybackup_<sanitized_url>.log`.
 
 - **`--progress`**:<br>
-Shows a progress bar instead of the default output.
+  Shows a progress bar instead of the default output.
 
 - **`--workers`** `<count>`:<br>
-Sets the number of simultaneous download workers. Default is 1, safe range is about 10. Be cautious as too many workers may lead to refused connections from the Wayback Machine.
+  Sets the number of simultaneous download workers. Default is 1, safe range is about 10. Be cautious as too many workers may lead to refused connections from the Wayback Machine.
 
 - **`--no-redirect`**:<br>
-Disables following redirects of snapshots. Useful for preventing timestamp-folder mismatches caused by Archive.org redirects.
-  
+  Disables following redirects of snapshots. Useful for preventing timestamp-folder mismatches caused by Archive.org redirects.
+
 - **`--retry`** `<attempts>`:<br>
-Specifies number of retry attempts for failed downloads.
+  Specifies number of retry attempts for failed downloads.
 
 - **`--delay`** `<seconds>`:<br>
-Specifies delay between download requests in seconds. Default is no delay (0).
+  Specifies delay between download requests in seconds. Default is no delay (0).
 
 <!-- - **`--convert-links`**:<br>
 If set, all links in the downloaded files will be converted to local links. This is useful for offline browsing. The links are converted to the local path structure. Show output with `--verbosity trace`. -->
 
-### Special:
+#### Job Handling:
 
 - **`--reset`**:  
   If set, the job will be reset, and any existing `cdx`, `db`, `csv` files will be **deleted**. This allows you to start the job from scratch without considering previously downloaded data.
@@ -113,47 +137,60 @@ If set, all links in the downloaded files will be converted to local links. This
 - **`--keep`**:  
   If set, all files will be kept after the job is finished. This includes the `cdx` and `db` file. Without this argument, they will be deleted if the job finished successfully.
 
-# Usage 
+<br>
+<br>
+
+## Usage
 
 ### Handling Interrupted Jobs
-When a job is interrupted (by any reason), `pywaybackup` is designed to resume the job from where it left off. It automatically detects existing job data (based on the URL and <u>**optional query parameters**</u> - including output directory) and resumes the process without requiring manual intervention. Here's how the tool handles different scenarios:
-
-- **Default Behavior:** 
-  - On restarting the same job (same URL, <u>**optional query parameters**</u>, and output directory), the tool will:
-    - Reuse the existing `.cdx` and `.db` files.
-    - Resume downloading snapshots from the last successful point.
-    - Skip previously downloaded files to save time and resources.
-
-- **Manual Reset with `--reset`:** 
-  - This command deletes any existing `.cdx` and `.db` files associated with the job and starts the process from scratch.
-  - Useful if:
-    - The previous data is corrupted.
-    - You want to re-query the snapshots without considering previously downloaded data.
-
-- **Preserving Job Data with `--keep`:** 
-  - Normally, `.cdx` and `.db` files are deleted after the job finishes successfully.
-  - Use `--keep` to retain these files for future use (e.g., re-analysis or extending the query later).
-
-> **Note1:** The resumption process only works if the output directory remains the same as the one used during the initial job.
-> 
-> **Note2:** `--reset` will NOT delete the already downloaded files for now. You have to remove them 'by hand'.
-  
-### Example
-
-1. Start downloading all available snapshots:<br>`waybackup -u https://example.com -a`
-2. Interrupt the process `CTRL+C`<br>
-3. The tool will detect the existing job data and resume downloading from the last completed point:<br>`waybackup -u https://example.com -a`
-> **Important:** `waybackup -u https://example.com -c` -> The tool will NOT resume because a necessary identifier-changed
-4. This deletes any existing .cdx and .db files associated with the job and starts the process from scratch:<br>`waybackup -u https://example.com -a --reset`
-5. This ensures all job-related files are kept for future use, such as re-analysis or extending the query later:<br>`waybackup -u https://example.com -a --keep`
-
-## Output path structure
+
+`pywaybackup` resumes interrupted jobs. The tool automatically continues from where it left off.
+
+- Detects existing `.cdx` and `.db` files in an `output dir` to resume downloading from the last successful point.
+- Compares `URL`, `mode`, and `optional query parameters` to ensure automatic resumption.
+- Skips previously downloaded files to save time.
+  > **Note:** Changing URL, mode selection, query parameters or output prevents automatic resumption.
+
+#### Resetting a Job (`--reset`)
+
+- Deletes `.cdx` and `.db` files and restarts the process from scratch.
+- Does **not** remove already downloaded files.
+- `waybackup -u https://example.com -a --reset`
+
+#### Keeping Job Data (`--keep`)
+
+- Normally, `.cdx` and `.db` files are deleted after a successful job.
+- `--keep` preserves them for future re-analysis or extending the query.
+- `waybackup -u https://example.com -a --keep`
+
+<br>
+<br>
+
+## Examples
+
+1. Download a specific single snapshot of all available files (starting from root):<br>
+   `waybackup -u https://example.com -a --start 20210101000000 --end 20210101000000`
+2. Download a specific single snapshot of all available files (starting from a subdirectory):<br>
+   `waybackup -u https://example.com/subdir1/subdir2/assets/ -a --start 20210101000000 --end 20210101000000`
+3. Download a specific single snapshot of the exact given URL (no subdirs):<br>
+   `waybackup -u https://example.com -a --start 20210101000000 --end 20210101000000 --explicit`
+4. Download all snapshots of all available files in the given range:<br>
+   `waybackup -u https://example.com -a --start 20210101000000 --end 20231122000000`
+
+<br>
+<br>
+
+## Output
+
+### Path Structure
 
 The output path is currently structured as follows by an example for the query:<br>
-`http://example.com/subdir1/subdir2/assets/`:
+`http://example.com/subdir1/subdir2/assets/`
 <br><br>
 For the first and last version (`-f` or `-l`):
-- The requested path will only include all files/folders starting from your query-path.
+
+- Will only include all files/folders starting from your query-path.
+
 ```
 your/path/waybackup_snapshots/
 └── the_root_of_your_query/ (example.com/)
@@ -164,8 +201,11 @@ your/path/waybackup_snapshots/
                 ├── style.css
                 ...
 ```
+
 For all versions (`-a`):
-- Will currently create a folder named as the root of your query. Inside this folder, you will find all timestamps and per timestamp the path you requested.
+
+- Will create a folder named as the root of your query. Inside this folder, you will find all timestamps and per timestamp the path you requested.
+
 ```
 your/path/waybackup_snapshots/
 └── the_root_of_your_query/ (example.com/)
@@ -184,7 +224,7 @@ your/path/waybackup_snapshots/
     ...
 ```
 
-## CSV Output
+### CSV
 
 Each snapshot is stored with the following keys/values. These are either stored in a sqlite database while the download is running or saved into a CSV file after the download is finished.
 
@@ -206,15 +246,33 @@ For download queries:
 ]
 ```
 
+### Log
+
+Verbose:
+
+```
+-----> Worker: 2 - Attempt: [1/1] Snapshot ID: [23/81]
+SUCCESS   -> 200 OK
+          -> URL:  https://web.archive.org/web/20240225193302id_/https://example.com/assets/css/custom-styles.css
+          -> FILE: /home/manjaro/Stuff/python-wayback-machine-downloader/waybackup_snapshots/example.com/20240225193302id_/assets/css/custom-styles.css
+```
+
+Non-verbose:
+
+```
+55/81 - W:2 - SUCCESS - 20240225193302 - https://example.com/assets/css/custom-styles.css
+```
+
 ### Debugging
 
 Exceptions will be written into `waybackup_error.log` (each run overwrites the file).
 
-### Known ToDos
-
-- [ ] currently there is no logic to handle if both a http and https version of a page is available
+<br>
+<br>
 
 ## Contributing
 
 I'm always happy for some feature requests to improve the usability of this tool.
 Feel free to give suggestions and report issues. Project is still far from being perfect.
+
+> Please PR from dev into dev.