dar-backup 0.7.1__py3-none-any.whl → 0.8.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
dar_backup/Changelog.md CHANGED
@@ -1,7 +1,33 @@
1
1
  <!-- markdownlint-disable MD024 -->
2
2
  # dar-backup Changelog
3
3
 
4
- ## v2-beta-0.6.21 - not yet released
4
+ ## v2-beta-0.8.0 - 2025-06-13
5
+
6
+ Github link: [v2-beta-0.8.0](https://github.com/per2jensen/dar-backup/tree/v2-beta-0.8.0/v2)
7
+
8
+ ### Added
9
+
10
+ - Modified clone dashboard generator to produce easier to read dashboard and be more robust.
11
+ - Dir_traversal sanitation: clean_log.py now only accepts files in configured log directory to `--file` option.
12
+
13
+ ## v2-beta-0.7.2 - 2025-06-07
14
+
15
+ Github link: [v2-beta-0.7.2](https://github.com/per2jensen/dar-backup/tree/v2-beta-0.7.2/v2)
16
+
17
+ ### Added
18
+
19
+ - Refactored build system, so all dependencies are kept in `pyproject.toml`. The dependencies are separated into dev, packaging and delivery phases.
20
+ - Use `build.sh` to setup pytest environment in Github workflow.
21
+
22
+ - Do the same to get a development environment going.
23
+
24
+ - Added 2 new optional params to control log file roll.
25
+ - Enrolling into [Snyk code checker](https://snyk.io/code-checker/) and learning how to work with it.
26
+
27
+ - Snyk helped pointing out vulnerable versions of some packages used.
28
+ - Input sanitation started, there is room for improvement.
29
+
30
+ ## v2-beta-0.7.1 - 2025-05-22
5
31
 
6
32
  Github link: [v2-beta-0.7.1](https://github.com/per2jensen/dar-backup/tree/v2-beta-0.7.1/v2)
7
33
 
@@ -24,9 +50,9 @@ Github link: [v2-beta-0.7.1](https://github.com/per2jensen/dar-backup/tree/v2-be
24
50
  - Action + program to capture cloning stats and store them in v2/doc directory. Includes a badge.
25
51
 
26
52
  -- annotate new daily max number of clones
27
- -- Celebration badge when number of clones numbers are hit (just for fun)
53
+ -- Celebration badge when certain clones numbers are hit (just for fun)
28
54
 
29
- - Action + program to generate 12weeks cloning dashboard (a PNG) with annotation
55
+ - Action + program to generate 12 weeks cloning dashboard (a PNG) with annotation
30
56
  - Tweaked the auto completion setup in .bashrc, it stopped working for me unknown reasons (needs some looking into)
31
57
  - --verbose now affects the startup banner. Now it is printed only if --verbose is given
32
58
 
dar_backup/README.md CHANGED
@@ -3,15 +3,16 @@
3
3
 
4
4
  **Reliable DAR backups, automated in clean Python**
5
5
 
6
- [![codecov](https://codecov.io/gh/per2jensen/dar-backup/branch/main/graph/badge.svg)](https://codecov.io/gh/per2jensen/dar-backup)
6
+ [![Codecov](https://codecov.io/gh/per2jensen/dar-backup/branch/main/graph/badge.svg)](https://codecov.io/gh/per2jensen/dar-backup)
7
+ [![Snyk Vuln findings](https://snyk.io/test/github/per2jensen/dar-backup/badge.svg)](https://snyk.io/test/github/per2jensen/dar-backup)
7
8
  ![CI](https://github.com/per2jensen/dar-backup/actions/workflows/py-tests.yml/badge.svg)
8
9
  [![PyPI version](https://img.shields.io/pypi/v/dar-backup.svg)](https://pypi.org/project/dar-backup/)
9
- [![PyPI total Downloads](https://img.shields.io/badge/dynamic/json?color=blue&label=Total%20Downloads&query=total&url=https%3A%2F%2Fraw.githubusercontent.com%2Fper2jensen%2Fdar-backup%2Fmain%2Fdownloads.json)](https://pypi.org/project/dar-backup/)
10
+ [![PyPI downloads](https://img.shields.io/badge/dynamic/json?color=blue&label=PyPI%20downloads&query=total&url=https%3A%2F%2Fraw.githubusercontent.com%2Fper2jensen%2Fdar-backup%2Fmain%2Fdownloads.json)](https://pypi.org/project/dar-backup/)
10
11
  [![# clones](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/per2jensen/dar-backup/main/v2/doc/badges/badge_clones.json)](https://github.com/per2jensen/dar-backup/blob/main/v2/doc/weekly_clones.png)
11
12
  [![Milestone](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/per2jensen/dar-backup/main/v2/doc/badges/milestone_badge.json)](https://github.com/per2jensen/dar-backup/blob/main/v2/doc/weekly_clones.png)
12
13
 
13
14
  The wonderful 'dar' [Disk Archiver](https://github.com/Edrusb/DAR) is used for
14
- the heavy lifting, together with [parchive](https://github.com/Parchive/par2cmdline) suite in these scripts.
15
+ the heavy lifting, together with the [parchive](https://github.com/Parchive/par2cmdline) suite in these scripts.
15
16
 
16
17
  This is the `Python` based [**version 2**](https://github.com/per2jensen/dar-backup/tree/main/v2) of `dar-backup`.
17
18
 
@@ -73,6 +74,7 @@ This is the `Python` based [**version 2**](https://github.com/per2jensen/dar-bac
73
74
  - [Skipping cache directories](#skipping-cache-directories)
74
75
  - [Progress bar + current directory](#progress-bar-and-current-directory)
75
76
  - [Shell Autocompletion](#shell-autocompletion)
77
+ - [Easy development setup](#easy-development-setup)
76
78
  - [Todo](#todo)
77
79
  - [Known Limitations / Edge Cases](#known-limitations--edge-cases)
78
80
  - [Reference](#reference)
@@ -207,17 +209,17 @@ Setup the demo configurations and show a few operations
207
209
  <br>
208
210
 
209
211
  ```bash
210
- # see reference section for options tweaking the install
212
+ # See reference section for options tweaking the install
211
213
  demo --install
212
214
 
213
215
  # create catalog database
214
216
  manager --create-db
215
217
 
216
- # do a FULL backup of the installed backup definition called `demo`
218
+ # FULL backup as defined in backup definition `demo`
217
219
  dar-backup --full-backup
218
220
 
219
- # list the contents of the backup
220
- dar-backup --list-contents $(dar-backup --list |tail -n 1 | cut -d " " -f1)
221
+ # List the contents of the backup
222
+ dar-backup --list-contents demo_FULL_$(date '+%F')
221
223
  ```
222
224
 
223
225
  <details>
@@ -259,7 +261,7 @@ Config file: /home/user/.config/dar-backup/dar-backup.conf
259
261
 
260
262
 
261
263
 
262
- (venv) $ dar-backup --list-contents $(dar-backup --list |tail -n 1 | cut -d " " -f1)
264
+ (venv) $ dar-backup --list-contents demo_FULL_$(date '+%F')
263
265
  ========== Startup Settings ==========
264
266
  dar-backup.py: 0.7.1
265
267
  dar path: /home/user/.local/dar/bin/dar
@@ -287,10 +289,10 @@ Config file: /home/user/.config/dar-backup/dar-backup.conf
287
289
  Perform a restore and show the restored files
288
290
 
289
291
  ```bash
290
- # restore all files in the backup
291
- dar-backup --restore $(dar-backup --list |tail -n 1 | cut -d " " -f1) --verbose
292
+ # Restore all files in the backup
293
+ dar-backup --restore demo_FULL_$(date '+%F') --verbose
292
294
 
293
- # prove the files have been restored to directory as configured
295
+ # Prove the files have been restored to directory as configured
294
296
  find $HOME/dar-backup/restore
295
297
  ```
296
298
 
@@ -299,7 +301,7 @@ find $HOME/dar-backup/restore
299
301
  <summary>🎯 --restore details</summary>
300
302
 
301
303
  ```bash
302
- (venv) $ dar-backup --verbose --restore $(dar-backup --list |tail -n 1 | cut -d " " -f1)
304
+ (venv) $ dar-backup --restore demo_FULL_$(date '+%F') --verbose
303
305
  ========== Startup Settings ==========
304
306
  dar-backup.py: 0.7.1
305
307
  dar path: /home/user/.local/dar/bin/dar
@@ -337,7 +339,7 @@ PAR2 enabled: True
337
339
 
338
340
  > ✅ **Next steps**
339
341
  >
340
- > Tinker with `demo's` options:
342
+ > Play with `demo's` options:
341
343
  >
342
344
  > - --root-dir (perhaps $HOME)
343
345
  > - --dir-to-backup (perhaps Pictures)
@@ -472,8 +474,9 @@ Please review the [Code of Conduct](https://github.com/per2jensen/dar-backup/blo
472
474
 
473
475
  ## Requirements
474
476
 
477
+ - A linux system
475
478
  - dar
476
- - par2
479
+ - parchive (par2)
477
480
  - python3
478
481
  - python3-venv
479
482
 
@@ -578,27 +581,25 @@ See section 15 and section 16 in the supplied "LICENSE" file.
578
581
 
579
582
  ### 2 - configuration
580
583
 
581
- The dar-backup [demo](#demo-options) application can be used to demo how `dar-backup` works.
582
- It creates some directories, installs a demo configuration file and puts a demo backup definition in place.
584
+ The dar-backup [installer](#installer-options) application can be used to setup the needed directories for `dar-backup` to work.
585
+ It creates necessary directories as prescribed in the config file and optionally creates manager databases.
583
586
 
584
- `demo` is non-destructive and stops if some of the default directories exist.
587
+ `installer` can also add configuration of shell auto completion.
585
588
 
586
- Run `demo`
589
+ Step 1:
587
590
 
588
- ```bash
589
- demo --install
590
- ```
591
+ Create a config file - [see details on config file](#config-file))
591
592
 
592
- The output is
593
+ Step 2:
593
594
 
594
- ```text
595
- Directories created: `/home/user/dar-backup/` and `/home/user/.config/dar-backup`
596
- Config file deployed to /home/user/.config/dar-backup/dar-backup.conf
597
- Default backup definition deployed to /home/user/.config/dar-backup/backup.d/default
598
- 1. Now run `manager --create-db` to create the catalog database.
599
- 2. Then you can run `dar-backup --full-backup` to create a backup.
600
- 3. List backups with `dar-backup --list`
601
- 4. List contents of a backup with `dar-backup --list-contents <backup-name>`
595
+ Create one or more backup definitions - [see details on backup definitions](#backup-definition-example)
596
+
597
+ Step 3:
598
+
599
+ Run the installer:
600
+
601
+ ```bash
602
+ installer --config <path to dar-backup.conf> --install-autocompletion
602
603
  ```
603
604
 
604
605
  ### 3 - generate catalog databases
@@ -613,9 +614,7 @@ manager --create-db
613
614
 
614
615
  ### 4 - give dar-backup a spin
615
616
 
616
- The `demo` application has put a demo [backup definition](#backup-definition-example) in place in BACKUP.D_DIR (see [config file](#config-file)).
617
-
618
- You are now ready to do backups as configured in the backup definition.
617
+ You are now ready to do backups as configured in your backup definition(s).
619
618
 
620
619
  Give `dar-backup`a spin:
621
620
 
@@ -625,6 +624,9 @@ dar-backup --full-backup --verbose
625
624
  # list backups
626
625
  dar-backup --list
627
626
 
627
+ # list contents of a dar backup
628
+ dar-backup --list-contents <TAB>... <choose a backup>
629
+
628
630
  # see some examples on usage
629
631
  dar-backup --examples
630
632
 
@@ -663,6 +665,9 @@ Tilde `~` and environment variables can be used in the paths for various file lo
663
665
  ```text
664
666
  [MISC]
665
667
  LOGFILE_LOCATION=~/.dar-backup.log
668
+ # optional parameters
669
+ # LOGFILE_MAX_BYTES = 26214400 # 25 MB max file size is default, change as neeeded
670
+ # LOGFILE_BACKUP_COUNT = 5 # 5 backup log files is default, change as needed
666
671
  MAX_SIZE_VERIFICATION_MB = 20
667
672
  MIN_SIZE_VERIFICATION_MB = 1
668
673
  NO_FILES_VERIFICATION = 5
@@ -681,6 +686,8 @@ TEST_RESTORE_DIR = /tmp/dar-backup/restore/
681
686
 
682
687
  [AGE]
683
688
  # age settings are in days
689
+ # `cleanup` script removes archives and their .par redundancy files if older than configured.
690
+ # `cleanup` does not remove FULL archives, unless specifically told to and a "y" is answered to "are you sure?".
684
691
  DIFF_AGE = 100
685
692
  INCR_AGE = 40
686
693
 
@@ -836,6 +843,12 @@ The name of the file is the name of the backup definition.
836
843
 
837
844
  You can use as many backup definitions as you need.
838
845
 
846
+ > Note 👉
847
+ >
848
+ > Environment variables and tilde (~) not allowed here. `dar` does not expand them.
849
+ >
850
+ > See [TODO](#todo)
851
+
839
852
  ```text
840
853
  # Switch to ordered selection mode, which means that the following
841
854
  # options will be considered top to bottom
@@ -1209,6 +1222,8 @@ These [.darrc](#darrc) settings make `dar` print the current directory being pro
1209
1222
 
1210
1223
  This is very useful in very long running jobs to get an indication that the backup is proceeding normally.
1211
1224
 
1225
+ The `dar` output is streamed to the `dar-backup-commands.log` file.
1226
+
1212
1227
  ### Separate log file for command output
1213
1228
 
1214
1229
  Dar-backup's log file is called `dar-backup.log`.
@@ -1370,8 +1385,54 @@ Then reload Zsh:
1370
1385
  source ~/.zshrc
1371
1386
  ```
1372
1387
 
1388
+ ## Easy development setup
1389
+
1390
+ It is very easy to have your own development environment.
1391
+
1392
+ ```bash
1393
+ git clone https://github.com/per2jensen/dar-backup.git
1394
+ cd dar-backup/v2
1395
+ ./build.py
1396
+ ```
1397
+
1398
+ This script:
1399
+
1400
+ - Creates a Python virtual environment called `venv`
1401
+ - pip install `hatch`
1402
+ - pip install the development environment as setup in pyproject.toml
1403
+
1404
+ --
1405
+
1406
+ ```text
1407
+ dev = [
1408
+ "pytest",
1409
+ "wheel>=0.45.1",
1410
+ "requests>=2.32.2",
1411
+ "coverage>=7.8.2",
1412
+ "pytest>=8.4.0",
1413
+ "pytest-cov>=6.1.1",
1414
+ "psutil>=7.0.0",
1415
+ "pytest-timeout>=2.4.0",
1416
+ "httpcore>=0.17.3",
1417
+ "h11>=0.16.0",
1418
+ "zipp>=3.19.1",
1419
+ "anyio>=4.4.0",
1420
+ "black>=25.1.0"]
1421
+ ```
1422
+
1423
+ ✅ Your environment is now ready to activate and test!
1424
+
1425
+ Activate and run the test suite:
1426
+
1427
+ ```bash
1428
+ source venv/bin/activate # activate the virtual env
1429
+ pytest # run the test suite
1430
+ ```
1431
+
1373
1432
  ## Todo
1374
1433
 
1434
+ - Perhaps look into pre-processing backup definitions. As `dar` does not expand env vars
1435
+ `dar-backup` could do so and feed the result to `dar`.
1375
1436
  - When run interactively, a progress bar during test and par2 generation would be nice.
1376
1437
  - Look into a way to move the .par2 files away from the `dar` slices, to maximize chance of good redundancy.
1377
1438
  - Add option to dar-backup to use the `dar` option `--fsa-scope none`
@@ -1536,12 +1597,12 @@ Sets up `dar-backup` according to provided config file.
1536
1597
  The installer creates the necessary backup catalog databases if `--create-db` is given.
1537
1598
 
1538
1599
  ```bash
1539
- --config Sets up `dar-backup`.
1540
- --create-db Create backup catalog databases. Add it to --config
1541
- --install-autocompletion Add bash or zsh auto completion - idempotent
1542
- --remove-autocompletion Remove the auto completion from bash or zsh
1600
+ --config Path to a config file. The configured directories will be created.
1601
+ --create-db Create backup catalog databases. Use this option with `--config`.
1602
+ --install-autocompletion Add bash or zsh auto completion - idempotent.
1603
+ --remove-autocompletion Remove the auto completion from bash or zsh.
1543
1604
  -v, --version Display version and licensing information.
1544
- -h, --help Displays usage info
1605
+ -h, --help Displays usage info.
1545
1606
  ```
1546
1607
 
1547
1608
  ### Demo options
@@ -1561,7 +1622,7 @@ Create directories:
1561
1622
  Sets up demo config files:
1562
1623
 
1563
1624
  - ~/.config/dar-backup/dar-backup.conf
1564
- - ~/.config/dar-backup/backup.d/default
1625
+ - ~/.config/dar-backup/backup.d/demo
1565
1626
 
1566
1627
  ```bash
1567
1628
  -i, --install Sets up `dar-backup`.
dar_backup/__about__.py CHANGED
@@ -1,4 +1,4 @@
1
- __version__ = "0.7.1"
1
+ __version__ = "0.8.0"
2
2
 
3
3
  __license__ = '''Licensed under GNU GENERAL PUBLIC LICENSE v3, see the supplied file "LICENSE" for details.
4
4
  THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW, not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
dar_backup/clean_log.py CHANGED
@@ -137,6 +137,19 @@ def main():
137
137
  args.file = [config_settings.logfile_location]
138
138
 
139
139
  for file_path in args.file:
140
+
141
+ if ".." in os.path.normpath(file_path).split(os.sep):
142
+ print(f"Error: Path traversal is not allowed: '{file_path}'")
143
+ sys.exit(1)
144
+
145
+ logfile_dir = os.path.dirname(os.path.realpath(config_settings.logfile_location))
146
+ resolved_path = os.path.realpath(file_path)
147
+
148
+ if not resolved_path.startswith(logfile_dir + os.sep):
149
+ print(f"Error: File is outside allowed directory: '{file_path}'")
150
+ sys.exit(1)
151
+
152
+ # Validate the file path type and existence
140
153
  if not isinstance(file_path, (str, bytes, os.PathLike)):
141
154
  print(f"Error: Invalid file path type: {file_path}")
142
155
  sys.exit(1)
dar_backup/cleanup.py CHANGED
@@ -41,6 +41,8 @@ from dar_backup.util import show_version
41
41
  from dar_backup.util import get_invocation_command_line
42
42
  from dar_backup.util import print_aligned_settings
43
43
  from dar_backup.util import backup_definition_completer, list_archive_completer
44
+ from dar_backup.util import is_safe_filename
45
+ from dar_backup.util import show_scriptname
44
46
 
45
47
  from dar_backup.command_runner import CommandRunner
46
48
  from dar_backup.command_runner import CommandResult
@@ -80,7 +82,7 @@ def delete_old_backups(backup_dir, age, backup_type, args, backup_definition=Non
80
82
  if file_date < cutoff_date:
81
83
  file_path = os.path.join(backup_dir, filename)
82
84
  try:
83
- os.remove(file_path)
85
+ is_safe_filename(file_path) and os.remove(file_path)
84
86
  logger.info(f"Deleted {backup_type} backup: {file_path}")
85
87
  archive_name = filename.split('.')[0]
86
88
  if not archive_name in archives_deleted:
@@ -109,7 +111,7 @@ def delete_archive(backup_dir, archive_name, args):
109
111
  if archive_regex.match(filename):
110
112
  file_path = os.path.join(backup_dir, filename)
111
113
  try:
112
- os.remove(file_path)
114
+ is_safe_filename(file_path) and os.remove(file_path)
113
115
  logger.info(f"Deleted archive slice: {file_path}")
114
116
  files_deleted = True
115
117
  except Exception as e:
@@ -127,7 +129,7 @@ def delete_archive(backup_dir, archive_name, args):
127
129
  if par2_regex.match(filename):
128
130
  file_path = os.path.join(backup_dir, filename)
129
131
  try:
130
- os.remove(file_path)
132
+ is_safe_filename(file_path) and os.remove(file_path)
131
133
  logger.info(f"Deleted PAR2 file: {file_path}")
132
134
  files_deleted = True
133
135
  except Exception as e:
@@ -215,13 +217,13 @@ def main():
215
217
 
216
218
  # command_output_log = os.path.join(config_settings.logfile_location.removesuffix("dar-backup.log"), "dar-backup-commands.log")
217
219
  command_output_log = config_settings.logfile_location.replace("dar-backup.log", "dar-backup-commands.log")
218
- logger = setup_logging(config_settings.logfile_location, command_output_log, args.log_level, args.log_stdout)
220
+ logger = setup_logging(config_settings.logfile_location, command_output_log, args.log_level, args.log_stdout, logfile_max_bytes=config_settings.logfile_max_bytes, logfile_backup_count=config_settings.logfile_backup_count)
219
221
  command_logger = get_logger(command_output_logger = True)
220
222
  runner = CommandRunner(logger=logger, command_logger=command_logger)
221
223
 
222
224
  start_msgs: List[Tuple[str, str]] = []
223
225
 
224
- start_msgs.append(("cleanup.py:", about.__version__))
226
+ start_msgs.append((f"{show_scriptname()}:", about.__version__))
225
227
 
226
228
  logger.info(f"START TIME: {start_time}")
227
229
  logger.debug(f"Command line: {get_invocation_command_line()}")
@@ -233,6 +235,8 @@ def main():
233
235
  start_msgs.append(("Config file:", args.config_file))
234
236
  args.verbose and start_msgs.append(("Backup dir:", config_settings.backup_dir))
235
237
  start_msgs.append(("Logfile:", config_settings.logfile_location))
238
+ args.verbose and start_msgs.append(("Logfile max size (bytes):", config_settings.logfile_max_bytes))
239
+ args.verbose and start_msgs.append(("Logfile backup count:", config_settings.logfile_backup_count))
236
240
  args.verbose and start_msgs.append(("--alternate-archive-dir:", args.alternate_archive_dir))
237
241
  args.verbose and start_msgs.append(("--cleanup-specific-archives:", args.cleanup_specific_archives))
238
242
 
@@ -261,7 +265,8 @@ def main():
261
265
  if "_FULL_" in archive_name:
262
266
  if not confirm_full_archive_deletion(archive_name, args.test_mode):
263
267
  continue
264
- logger.info(f"Deleting archive: {archive_name}")
268
+ archive_path = os.path.join(config_settings.backup_dir, archive_name.strip())
269
+ logger.info(f"Deleting archive: {archive_path}")
265
270
  delete_archive(config_settings.backup_dir, archive_name.strip(), args)
266
271
  elif args.list:
267
272
  list_backups(config_settings.backup_dir, args.backup_definition)
@@ -44,6 +44,8 @@ class ConfigSettings:
44
44
  incr_age: int = field(init=False)
45
45
  error_correction_percent: int = field(init=False)
46
46
  par2_enabled: bool = field(init=False)
47
+ logfile_max_bytes: int = field(init=False)
48
+ logfile_no_count: int = field(init=False)
47
49
 
48
50
 
49
51
  OPTIONAL_CONFIG_FIELDS = [
@@ -54,6 +56,20 @@ class ConfigSettings:
54
56
  "type": str,
55
57
  "default": None,
56
58
  },
59
+ {
60
+ "section": "MISC",
61
+ "key": "LOGFILE_MAX_BYTES",
62
+ "attr": "logfile_max_bytes",
63
+ "type": int,
64
+ "default": 26214400 , # 25 MB
65
+ },
66
+ {
67
+ "section": "MISC",
68
+ "key": "LOGFILE_BACKUP_COUNT",
69
+ "attr": "logfile_backup_count",
70
+ "type": int,
71
+ "default": 5,
72
+ },
57
73
  # Add more optional fields here
58
74
  ]
59
75
 
@@ -26,6 +26,10 @@
26
26
 
27
27
  [MISC]
28
28
  LOGFILE_LOCATION = {{ vars_map.DAR_BACKUP_DIR -}}/dar-backup.log
29
+ # optional parameters
30
+ # LOGFILE_MAX_BYTES = 26214400 # 25 MB default, change as neeeded
31
+ # LOGFILE_BACKUP_COUNT = 5 # default, change as needed
32
+
29
33
  MAX_SIZE_VERIFICATION_MB = 2
30
34
  MIN_SIZE_VERIFICATION_MB = 0
31
35
  NO_FILES_VERIFICATION = 1
dar_backup/dar_backup.py CHANGED
@@ -55,6 +55,8 @@ from dar_backup.util import get_invocation_command_line
55
55
  from dar_backup.util import get_binary_info
56
56
  from dar_backup.util import print_aligned_settings
57
57
  from dar_backup.util import backup_definition_completer, list_archive_completer
58
+ from dar_backup.util import show_scriptname
59
+ from dar_backup.util import print_debug
58
60
 
59
61
  from dar_backup.command_runner import CommandRunner
60
62
  from dar_backup.command_runner import CommandResult
@@ -92,6 +94,7 @@ def generic_backup(type: str, command: List[str], backup_file: str, backup_defin
92
94
  Returns:
93
95
  List of tuples (<msg>, <exit_code>) of errors not considered critical enough for raising an exception
94
96
  """
97
+
95
98
  result: List[tuple] = []
96
99
 
97
100
  logger.info(f"===> Starting {type} backup for {backup_definition}")
@@ -105,7 +108,6 @@ def generic_backup(type: str, command: List[str], backup_file: str, backup_defin
105
108
  stop_event = Event()
106
109
  session_marker = f"=== START BACKUP SESSION: {int(time())} ==="
107
110
  get_logger(command_output_logger=True).info(session_marker)
108
-
109
111
  progress_thread = threading.Thread(
110
112
  target=show_log_driven_bar,
111
113
  args=(log_path, stop_event, session_marker),
@@ -534,12 +536,12 @@ def perform_backup(args: argparse.Namespace, config_settings: ConfigSettings, ba
534
536
 
535
537
  # Perform backup
536
538
  backup_result = generic_backup(backup_type, command, backup_file, backup_definition_path, args.darrc, config_settings, args)
539
+
537
540
  if not isinstance(backup_result, list) or not all(isinstance(i, tuple) and len(i) == 2 for i in backup_result):
538
541
  logger.error("Unexpected return format from generic_backup")
539
542
  backup_result = [("Unexpected return format from generic_backup", 1)]
540
543
 
541
544
  results.extend(backup_result)
542
-
543
545
  logger.info("Starting verification...")
544
546
  verify_result = verify(args, backup_file, backup_definition_path, config_settings)
545
547
  if verify_result:
@@ -838,7 +840,7 @@ def main():
838
840
  if command_output_log == config_settings.logfile_location:
839
841
  print(f"Error: logfile_location in {args.config_file} does not end at 'dar-backup.log', exiting", file=stderr)
840
842
 
841
- logger = setup_logging(config_settings.logfile_location, command_output_log, args.log_level, args.log_stdout)
843
+ logger = setup_logging(config_settings.logfile_location, command_output_log, args.log_level, args.log_stdout, logfile_max_bytes=config_settings.logfile_max_bytes, logfile_backup_count=config_settings.logfile_backup_count)
842
844
  command_logger = get_logger(command_output_logger = True)
843
845
  runner = CommandRunner(logger=logger, command_logger=command_logger)
844
846
 
@@ -863,11 +865,11 @@ def main():
863
865
  start_msgs: List[Tuple[str, str]] = []
864
866
 
865
867
  start_time=int(time())
866
- start_msgs.append(('dar-backup.py:', about.__version__))
868
+ start_msgs.append((f"{show_scriptname()}:", about.__version__))
867
869
  logger.info(f"START TIME: {start_time}")
868
- logger.debug(f"Command line: {get_invocation_command_line()}")
869
- logger.debug(f"{'`Args`:\n'}{args}")
870
- logger.debug(f"{'`Config_settings`:\n'}{config_settings}")
870
+ logger.debug(f"Command line:\n{get_invocation_command_line()}")
871
+ logger.debug(f"`Args`:\n{args}")
872
+ logger.debug(f"`Config_settings`:\n{config_settings}")
871
873
  dar_properties = get_binary_info(command='dar')
872
874
  start_msgs.append(('dar path:', dar_properties['path']))
873
875
  start_msgs.append(('dar version:', dar_properties['version']))
@@ -890,6 +892,9 @@ def main():
890
892
  args.verbose and start_msgs.append(("Restore dir:", restore_dir))
891
893
 
892
894
  args.verbose and start_msgs.append(("Logfile location:", config_settings.logfile_location))
895
+ args.verbose and start_msgs.append(("Logfile max size (bytes):", config_settings.logfile_max_bytes))
896
+ args.verbose and start_msgs.append(("Logfile backup count:", config_settings.logfile_backup_count))
897
+
893
898
  args.verbose and start_msgs.append(("PAR2 enabled:", config_settings.par2_enabled))
894
899
  args.verbose and start_msgs.append(("--do-not-compare:", args.do_not_compare))
895
900
 
@@ -928,6 +933,7 @@ def main():
928
933
 
929
934
  requirements('POSTREQ', config_settings)
930
935
 
936
+
931
937
  except Exception as e:
932
938
  logger.error("Exception details:", exc_info=True)
933
939
  results.append((repr(e), 1))
dar_backup/installer.py CHANGED
@@ -12,6 +12,7 @@ from dar_backup.manager import create_db
12
12
  # Always expand manager DB dir correctly, using helper function
13
13
  from dar_backup.manager import get_db_dir
14
14
  from dar_backup.util import expand_path
15
+ from dar_backup.util import is_safe_path
15
16
 
16
17
  def install_autocompletion():
17
18
  """Detect user shell, choose RC file, and idempotently append autocompletion."""
@@ -99,7 +100,7 @@ def uninstall_autocompletion() -> str:
99
100
 
100
101
 
101
102
 
102
- def run_installer(config_file: str, create_db_flag: bool, install_ac_flag: bool):
103
+ def run_installer(config_file: str, create_db_flag: bool):
103
104
  """
104
105
  Run the installation process for dar-backup using the given config file.
105
106
 
@@ -116,6 +117,8 @@ def run_installer(config_file: str, create_db_flag: bool, install_ac_flag: bool)
116
117
  config_file = os.path.expanduser(os.path.expandvars(config_file))
117
118
  config_settings = ConfigSettings(config_file)
118
119
 
120
+ print(f"Using config settings: {config_settings}")
121
+
119
122
  # Set up logging
120
123
  command_log = config_settings.logfile_location.replace("dar-backup.log", "dar-backup-commands.log")
121
124
  logger = setup_logging(
@@ -137,6 +140,9 @@ def run_installer(config_file: str, create_db_flag: bool, install_ac_flag: bool)
137
140
  }
138
141
 
139
142
  for name, dir_path in required_dirs.items():
143
+ if not is_safe_path(dir_path):
144
+ logger.error(f"Unsafe path detected: {dir_path} ({name})")
145
+ raise ValueError(f"Unsafe path detected: {dir_path} ({name})")
140
146
  expanded = Path(expand_path(dir_path))
141
147
  if not expanded.exists():
142
148
  logger.info(f"Creating directory: {expanded} ({name})")
@@ -149,12 +155,11 @@ def run_installer(config_file: str, create_db_flag: bool, install_ac_flag: bool)
149
155
  print(f"Creating catalog for: {backup_def}")
150
156
  result = create_db(backup_def, config_settings, logger, runner)
151
157
  if result == 0:
152
- print(f"✔️ Catalog created (or already existed): {backup_def}")
158
+ print(f"✔️ Catalog created (or already exist): {backup_def}")
153
159
  else:
154
160
  print(f"❌ Failed to create catalog: {backup_def}")
155
161
 
156
162
 
157
-
158
163
  def main():
159
164
  parser = argparse.ArgumentParser(description="dar-backup installer")
160
165
  parser.add_argument("--config", required=False, help="Path to config file")
@@ -177,12 +182,17 @@ def main():
177
182
 
178
183
 
179
184
  if args.config:
185
+ if not os.path.exists(args.config):
186
+ print(f"❌ Config file does not exist: {args.config}")
187
+ return
180
188
  run_installer(args.config, args.create_db)
181
- elif args.install_autocompletion:
189
+
190
+ if args.install_autocompletion:
182
191
  install_autocompletion()
183
192
  elif args.remove_autocompletion:
184
193
  uninstall_autocompletion()
185
194
 
195
+
186
196
 
187
197
  if __name__ == "__main__":
188
198
  main()
dar_backup/manager.py CHANGED
@@ -40,6 +40,7 @@ from dar_backup.util import get_binary_info
40
40
  from dar_backup.util import show_version
41
41
  from dar_backup.util import get_invocation_command_line
42
42
  from dar_backup.util import print_aligned_settings
43
+ from dar_backup.util import show_scriptname
43
44
 
44
45
  from dar_backup.command_runner import CommandRunner
45
46
  from dar_backup.command_runner import CommandResult
@@ -544,14 +545,15 @@ def main():
544
545
  return
545
546
 
546
547
  command_output_log = config_settings.logfile_location.replace("dar-backup.log", "dar-backup-commands.log")
547
- logger = setup_logging(config_settings.logfile_location, command_output_log, args.log_level, args.log_stdout)
548
+ logger = setup_logging(config_settings.logfile_location, command_output_log, args.log_level, args.log_stdout, logfile_max_bytes=config_settings.logfile_max_bytes, logfile_backup_count=config_settings.logfile_backup_count)
548
549
  command_logger = get_logger(command_output_logger=True)
549
550
  runner = CommandRunner(logger=logger, command_logger=command_logger)
550
551
 
551
552
  start_msgs: List[Tuple[str, str]] = []
552
553
 
553
554
  start_time = int(time())
554
- start_msgs.append((f"{SCRIPTNAME}:", about.__version__))
555
+
556
+ start_msgs.append((f"{show_scriptname()}:", about.__version__))
555
557
  logger.info(f"START TIME: {start_time}")
556
558
  logger.debug(f"Command line: {get_invocation_command_line()}")
557
559
  logger.debug(f"`args`:\n{args}")
@@ -559,6 +561,8 @@ def main():
559
561
  start_msgs.append(("Config file:", args.config_file))
560
562
  args.verbose and start_msgs.append(("Backup dir:", config_settings.backup_dir))
561
563
  start_msgs.append(("Logfile:", config_settings.logfile_location))
564
+ args.verbose and start_msgs.append(("Logfile max size (bytes):", config_settings.logfile_max_bytes))
565
+ args.verbose and start_msgs.append(("Logfile backup count:", config_settings.logfile_backup_count))
562
566
  args.verbose and start_msgs.append(("--alternate-archive-dir:", args.alternate_archive_dir))
563
567
  args.verbose and start_msgs.append(("--remove-specific-archive:", args.remove_specific_archive))
564
568
  dar_manager_properties = get_binary_info(command='dar_manager')
@@ -102,3 +102,4 @@ def show_log_driven_bar(log_path: str, stop_event: Event, session_marker: str, m
102
102
  if stop_event.is_set():
103
103
  break
104
104
 
105
+ # Rich prints a \n here, I will live with it
dar_backup/util.py CHANGED
@@ -12,7 +12,9 @@ See section 15 and section 16 in the supplied "LICENSE" file
12
12
  import typing
13
13
  import locale
14
14
  import configparser
15
+ import inspect
15
16
  import logging
17
+
16
18
  import os
17
19
  import re
18
20
  import subprocess
@@ -28,6 +30,7 @@ import dar_backup.__about__ as about
28
30
  from argcomplete.completers import ChoicesCompleter
29
31
  from datetime import datetime
30
32
  from dar_backup.config_settings import ConfigSettings
33
+ from logging.handlers import RotatingFileHandler
31
34
  from pathlib import Path
32
35
  from rich.console import Console
33
36
  from rich.text import Text
@@ -39,7 +42,16 @@ from typing import Tuple
39
42
  logger=None
40
43
  secondary_logger=None
41
44
 
42
- def setup_logging(log_file: str, command_output_log_file: str, log_level: str = "info", log_to_stdout: bool = False) -> logging.Logger:
45
+ #def setup_logging(log_file: str, command_output_log_file: str, log_level: str = "info", log_to_stdout: bool = False) -> logging.Logger:
46
+ def setup_logging(
47
+ log_file: str,
48
+ command_output_log_file: str,
49
+ log_level: str = "info",
50
+ log_to_stdout: bool = False,
51
+ logfile_max_bytes: int = 26214400,
52
+ logfile_backup_count: int = 5,
53
+ ) -> logging.Logger:
54
+
43
55
  """
44
56
  Sets up logging for the main program and a separate secondary logfile for command outputs.
45
57
 
@@ -48,9 +60,11 @@ def setup_logging(log_file: str, command_output_log_file: str, log_level: str =
48
60
  command_output_log_file (str): The path to the secondary log file for command outputs.
49
61
  log_level (str): The log level to use. Can be "info", "debug", or "trace". Defaults to "info".
50
62
  log_to_stdout (bool): If True, log messages will be printed to the console. Defaults to False.
63
+ logfile_max_bytes: max file size of a log file, defailt = 26214400.
64
+ logfile_backup_count: max numbers of logs files, default = 5.
51
65
 
52
66
  Returns:
53
- None
67
+ a RotatingFileHandler logger instance.
54
68
 
55
69
  Raises:
56
70
  Exception: If an error occurs during logging initialization
@@ -66,20 +80,34 @@ def setup_logging(log_file: str, command_output_log_file: str, log_level: str =
66
80
 
67
81
  logging.Logger.trace = trace
68
82
 
83
+ file_handler = RotatingFileHandler(
84
+ log_file,
85
+ maxBytes=logfile_max_bytes,
86
+ backupCount=logfile_backup_count,
87
+ encoding="utf-8",
88
+ )
89
+
90
+ command_handler = RotatingFileHandler(
91
+ command_output_log_file,
92
+ maxBytes=logfile_max_bytes,
93
+ backupCount=logfile_backup_count,
94
+ encoding="utf-8",
95
+ )
96
+
97
+ formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
98
+ file_handler.setFormatter(formatter)
99
+ command_handler.setFormatter(formatter)
100
+
101
+
69
102
  # Setup main logger
70
103
  logger = logging.getLogger("main_logger")
71
104
  logger.setLevel(logging.DEBUG if log_level == "debug" else TRACE_LEVEL_NUM if log_level == "trace" else logging.INFO)
72
- formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
73
- file_handler = logging.FileHandler(log_file)
74
- file_handler.setFormatter(formatter)
75
105
  logger.addHandler(file_handler)
76
106
 
77
107
  # Setup secondary logger for command outputs
78
108
  secondary_logger = logging.getLogger("command_output_logger")
79
109
  secondary_logger.setLevel(logging.DEBUG if log_level == "debug" else TRACE_LEVEL_NUM if log_level == "trace" else logging.INFO)
80
- sec_file_handler = logging.FileHandler(command_output_log_file)
81
- sec_file_handler.setFormatter(formatter)
82
- secondary_logger.addHandler(sec_file_handler)
110
+ secondary_logger.addHandler(command_handler)
83
111
 
84
112
  if log_to_stdout:
85
113
  stdout_handler = logging.StreamHandler(sys.stdout)
@@ -124,6 +152,15 @@ completer_logger = _setup_completer_logger()
124
152
  completer_logger.debug("Completer logger initialized.")
125
153
 
126
154
 
155
+ def print_debug(msg):
156
+ """
157
+ Print a debug message with the filename and line number of the caller.
158
+ """
159
+ frame = inspect.currentframe().f_back
160
+ print(f"[DEBUG] {frame.f_code.co_filename}:{frame.f_lineno} - {repr(msg)}")
161
+
162
+
163
+
127
164
  def get_invocation_command_line() -> str:
128
165
  """
129
166
  Safely retrieves the exact command line used to invoke the current Python process.
@@ -146,6 +183,17 @@ def get_invocation_command_line() -> str:
146
183
  return f"[error: could not read /proc/[pid]/cmdline: {e}]"
147
184
 
148
185
 
186
+ def show_scriptname() -> str:
187
+ """
188
+ Return script name, useful in start banner for example
189
+ """
190
+ try:
191
+ scriptname = os.path.basename(sys.argv[0])
192
+ except:
193
+ scriptname = "unknown"
194
+ return scriptname
195
+
196
+
149
197
  def show_version():
150
198
  script_name = os.path.basename(sys.argv[0])
151
199
  print(f"{script_name} {about.__version__}")
@@ -683,3 +731,30 @@ def normalize_dir(path: str) -> str:
683
731
  normalized = str(p)
684
732
  return normalized
685
733
 
734
+
735
+
736
+ # Reusable pattern for archive file naming
737
+ archive_pattern = re.compile(
738
+ r'^.+?_(FULL|DIFF|INCR)_(\d{4}-\d{2}-\d{2})\.\d+\.dar'
739
+ r'(?:\.vol\d+(?:\+\d+)?\.par2|\.par2)?$'
740
+ )
741
+
742
+ def is_safe_filename(filename: str) -> bool:
743
+ """
744
+ Validates that the filename matches acceptable dar/par2 naming convention.
745
+ """
746
+ return archive_pattern.match(filename) is not None
747
+
748
+ def is_safe_path(path: str) -> bool:
749
+ """
750
+ Validates that the full path is absolute, has no '..'.
751
+ """
752
+ normalized = os.path.normpath(path)
753
+ filename = os.path.basename(normalized)
754
+
755
+ return (
756
+ os.path.isabs(normalized)
757
+ and '..' not in normalized.split(os.sep)
758
+ )
759
+
760
+
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: dar-backup
3
- Version: 0.7.1
3
+ Version: 0.8.0
4
4
  Summary: A script to do full, differential and incremental backups using dar. Some files are restored from the backups during verification, after which par2 redundancy files are created. The script also has a cleanup feature to remove old backups and par2 files.
5
5
  Project-URL: GPG Public Key, https://keys.openpgp.org/search?q=dar-backup@pm.me
6
6
  Project-URL: Homepage, https://github.com/per2jensen/dar-backup/tree/main/v2
@@ -693,6 +693,27 @@ Requires-Dist: argcomplete>=3.6.2
693
693
  Requires-Dist: inputimeout>=1.0.4
694
694
  Requires-Dist: jinja2>=3.1.6
695
695
  Requires-Dist: rich>=13.0.0
696
+ Provides-Extra: dev
697
+ Requires-Dist: anyio>=4.4.0; extra == 'dev'
698
+ Requires-Dist: black>=25.1.0; extra == 'dev'
699
+ Requires-Dist: coverage>=7.8.2; extra == 'dev'
700
+ Requires-Dist: h11>=0.16.0; extra == 'dev'
701
+ Requires-Dist: httpcore>=0.17.3; extra == 'dev'
702
+ Requires-Dist: matplotlib>=3.10.3; extra == 'dev'
703
+ Requires-Dist: pandas>=2.3.0; extra == 'dev'
704
+ Requires-Dist: psutil>=7.0.0; extra == 'dev'
705
+ Requires-Dist: pytest; extra == 'dev'
706
+ Requires-Dist: pytest-cov>=6.1.1; extra == 'dev'
707
+ Requires-Dist: pytest-timeout>=2.4.0; extra == 'dev'
708
+ Requires-Dist: pytest>=8.4.0; extra == 'dev'
709
+ Requires-Dist: requests>=2.32.2; extra == 'dev'
710
+ Requires-Dist: wheel>=0.45.1; extra == 'dev'
711
+ Requires-Dist: zipp>=3.19.1; extra == 'dev'
712
+ Provides-Extra: packaging
713
+ Requires-Dist: build>=1.2.2; extra == 'packaging'
714
+ Requires-Dist: hatch>=1.14.1; extra == 'packaging'
715
+ Requires-Dist: hatchling>=1.27.0; extra == 'packaging'
716
+ Requires-Dist: twine>=6.1.0; extra == 'packaging'
696
717
  Description-Content-Type: text/markdown
697
718
 
698
719
  <!-- markdownlint-disable MD024 -->
@@ -700,15 +721,16 @@ Description-Content-Type: text/markdown
700
721
 
701
722
  **Reliable DAR backups, automated in clean Python**
702
723
 
703
- [![codecov](https://codecov.io/gh/per2jensen/dar-backup/branch/main/graph/badge.svg)](https://codecov.io/gh/per2jensen/dar-backup)
724
+ [![Codecov](https://codecov.io/gh/per2jensen/dar-backup/branch/main/graph/badge.svg)](https://codecov.io/gh/per2jensen/dar-backup)
725
+ [![Snyk Vuln findings](https://snyk.io/test/github/per2jensen/dar-backup/badge.svg)](https://snyk.io/test/github/per2jensen/dar-backup)
704
726
  ![CI](https://github.com/per2jensen/dar-backup/actions/workflows/py-tests.yml/badge.svg)
705
727
  [![PyPI version](https://img.shields.io/pypi/v/dar-backup.svg)](https://pypi.org/project/dar-backup/)
706
- [![PyPI total Downloads](https://img.shields.io/badge/dynamic/json?color=blue&label=Total%20Downloads&query=total&url=https%3A%2F%2Fraw.githubusercontent.com%2Fper2jensen%2Fdar-backup%2Fmain%2Fdownloads.json)](https://pypi.org/project/dar-backup/)
728
+ [![PyPI downloads](https://img.shields.io/badge/dynamic/json?color=blue&label=PyPI%20downloads&query=total&url=https%3A%2F%2Fraw.githubusercontent.com%2Fper2jensen%2Fdar-backup%2Fmain%2Fdownloads.json)](https://pypi.org/project/dar-backup/)
707
729
  [![# clones](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/per2jensen/dar-backup/main/v2/doc/badges/badge_clones.json)](https://github.com/per2jensen/dar-backup/blob/main/v2/doc/weekly_clones.png)
708
730
  [![Milestone](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/per2jensen/dar-backup/main/v2/doc/badges/milestone_badge.json)](https://github.com/per2jensen/dar-backup/blob/main/v2/doc/weekly_clones.png)
709
731
 
710
732
  The wonderful 'dar' [Disk Archiver](https://github.com/Edrusb/DAR) is used for
711
- the heavy lifting, together with [parchive](https://github.com/Parchive/par2cmdline) suite in these scripts.
733
+ the heavy lifting, together with the [parchive](https://github.com/Parchive/par2cmdline) suite in these scripts.
712
734
 
713
735
  This is the `Python` based [**version 2**](https://github.com/per2jensen/dar-backup/tree/main/v2) of `dar-backup`.
714
736
 
@@ -770,6 +792,7 @@ This is the `Python` based [**version 2**](https://github.com/per2jensen/dar-bac
770
792
  - [Skipping cache directories](#skipping-cache-directories)
771
793
  - [Progress bar + current directory](#progress-bar-and-current-directory)
772
794
  - [Shell Autocompletion](#shell-autocompletion)
795
+ - [Easy development setup](#easy-development-setup)
773
796
  - [Todo](#todo)
774
797
  - [Known Limitations / Edge Cases](#known-limitations--edge-cases)
775
798
  - [Reference](#reference)
@@ -904,17 +927,17 @@ Setup the demo configurations and show a few operations
904
927
  <br>
905
928
 
906
929
  ```bash
907
- # see reference section for options tweaking the install
930
+ # See reference section for options tweaking the install
908
931
  demo --install
909
932
 
910
933
  # create catalog database
911
934
  manager --create-db
912
935
 
913
- # do a FULL backup of the installed backup definition called `demo`
936
+ # FULL backup as defined in backup definition `demo`
914
937
  dar-backup --full-backup
915
938
 
916
- # list the contents of the backup
917
- dar-backup --list-contents $(dar-backup --list |tail -n 1 | cut -d " " -f1)
939
+ # List the contents of the backup
940
+ dar-backup --list-contents demo_FULL_$(date '+%F')
918
941
  ```
919
942
 
920
943
  <details>
@@ -956,7 +979,7 @@ Config file: /home/user/.config/dar-backup/dar-backup.conf
956
979
 
957
980
 
958
981
 
959
- (venv) $ dar-backup --list-contents $(dar-backup --list |tail -n 1 | cut -d " " -f1)
982
+ (venv) $ dar-backup --list-contents demo_FULL_$(date '+%F')
960
983
  ========== Startup Settings ==========
961
984
  dar-backup.py: 0.7.1
962
985
  dar path: /home/user/.local/dar/bin/dar
@@ -984,10 +1007,10 @@ Config file: /home/user/.config/dar-backup/dar-backup.conf
984
1007
  Perform a restore and show the restored files
985
1008
 
986
1009
  ```bash
987
- # restore all files in the backup
988
- dar-backup --restore $(dar-backup --list |tail -n 1 | cut -d " " -f1) --verbose
1010
+ # Restore all files in the backup
1011
+ dar-backup --restore demo_FULL_$(date '+%F') --verbose
989
1012
 
990
- # prove the files have been restored to directory as configured
1013
+ # Prove the files have been restored to directory as configured
991
1014
  find $HOME/dar-backup/restore
992
1015
  ```
993
1016
 
@@ -996,7 +1019,7 @@ find $HOME/dar-backup/restore
996
1019
  <summary>🎯 --restore details</summary>
997
1020
 
998
1021
  ```bash
999
- (venv) $ dar-backup --verbose --restore $(dar-backup --list |tail -n 1 | cut -d " " -f1)
1022
+ (venv) $ dar-backup --restore demo_FULL_$(date '+%F') --verbose
1000
1023
  ========== Startup Settings ==========
1001
1024
  dar-backup.py: 0.7.1
1002
1025
  dar path: /home/user/.local/dar/bin/dar
@@ -1034,7 +1057,7 @@ PAR2 enabled: True
1034
1057
 
1035
1058
  > ✅ **Next steps**
1036
1059
  >
1037
- > Tinker with `demo's` options:
1060
+ > Play with `demo's` options:
1038
1061
  >
1039
1062
  > - --root-dir (perhaps $HOME)
1040
1063
  > - --dir-to-backup (perhaps Pictures)
@@ -1169,8 +1192,9 @@ Please review the [Code of Conduct](https://github.com/per2jensen/dar-backup/blo
1169
1192
 
1170
1193
  ## Requirements
1171
1194
 
1195
+ - A linux system
1172
1196
  - dar
1173
- - par2
1197
+ - parchive (par2)
1174
1198
  - python3
1175
1199
  - python3-venv
1176
1200
 
@@ -1275,27 +1299,25 @@ See section 15 and section 16 in the supplied "LICENSE" file.
1275
1299
 
1276
1300
  ### 2 - configuration
1277
1301
 
1278
- The dar-backup [demo](#demo-options) application can be used to demo how `dar-backup` works.
1279
- It creates some directories, installs a demo configuration file and puts a demo backup definition in place.
1302
+ The dar-backup [installer](#installer-options) application can be used to setup the needed directories for `dar-backup` to work.
1303
+ It creates necessary directories as prescribed in the config file and optionally creates manager databases.
1280
1304
 
1281
- `demo` is non-destructive and stops if some of the default directories exist.
1305
+ `installer` can also add configuration of shell auto completion.
1282
1306
 
1283
- Run `demo`
1307
+ Step 1:
1284
1308
 
1285
- ```bash
1286
- demo --install
1287
- ```
1309
+ Create a config file - [see details on config file](#config-file))
1288
1310
 
1289
- The output is
1311
+ Step 2:
1290
1312
 
1291
- ```text
1292
- Directories created: `/home/user/dar-backup/` and `/home/user/.config/dar-backup`
1293
- Config file deployed to /home/user/.config/dar-backup/dar-backup.conf
1294
- Default backup definition deployed to /home/user/.config/dar-backup/backup.d/default
1295
- 1. Now run `manager --create-db` to create the catalog database.
1296
- 2. Then you can run `dar-backup --full-backup` to create a backup.
1297
- 3. List backups with `dar-backup --list`
1298
- 4. List contents of a backup with `dar-backup --list-contents <backup-name>`
1313
+ Create one or more backup definitions - [see details on backup definitions](#backup-definition-example)
1314
+
1315
+ Step 3:
1316
+
1317
+ Run the installer:
1318
+
1319
+ ```bash
1320
+ installer --config <path to dar-backup.conf> --install-autocompletion
1299
1321
  ```
1300
1322
 
1301
1323
  ### 3 - generate catalog databases
@@ -1310,9 +1332,7 @@ manager --create-db
1310
1332
 
1311
1333
  ### 4 - give dar-backup a spin
1312
1334
 
1313
- The `demo` application has put a demo [backup definition](#backup-definition-example) in place in BACKUP.D_DIR (see [config file](#config-file)).
1314
-
1315
- You are now ready to do backups as configured in the backup definition.
1335
+ You are now ready to do backups as configured in your backup definition(s).
1316
1336
 
1317
1337
  Give `dar-backup`a spin:
1318
1338
 
@@ -1322,6 +1342,9 @@ dar-backup --full-backup --verbose
1322
1342
  # list backups
1323
1343
  dar-backup --list
1324
1344
 
1345
+ # list contents of a dar backup
1346
+ dar-backup --list-contents <TAB>... <choose a backup>
1347
+
1325
1348
  # see some examples on usage
1326
1349
  dar-backup --examples
1327
1350
 
@@ -1360,6 +1383,9 @@ Tilde `~` and environment variables can be used in the paths for various file lo
1360
1383
  ```text
1361
1384
  [MISC]
1362
1385
  LOGFILE_LOCATION=~/.dar-backup.log
1386
+ # optional parameters
1387
+ # LOGFILE_MAX_BYTES = 26214400 # 25 MB max file size is default, change as neeeded
1388
+ # LOGFILE_BACKUP_COUNT = 5 # 5 backup log files is default, change as needed
1363
1389
  MAX_SIZE_VERIFICATION_MB = 20
1364
1390
  MIN_SIZE_VERIFICATION_MB = 1
1365
1391
  NO_FILES_VERIFICATION = 5
@@ -1378,6 +1404,8 @@ TEST_RESTORE_DIR = /tmp/dar-backup/restore/
1378
1404
 
1379
1405
  [AGE]
1380
1406
  # age settings are in days
1407
+ # `cleanup` script removes archives and their .par redundancy files if older than configured.
1408
+ # `cleanup` does not remove FULL archives, unless specifically told to and a "y" is answered to "are you sure?".
1381
1409
  DIFF_AGE = 100
1382
1410
  INCR_AGE = 40
1383
1411
 
@@ -1533,6 +1561,12 @@ The name of the file is the name of the backup definition.
1533
1561
 
1534
1562
  You can use as many backup definitions as you need.
1535
1563
 
1564
+ > Note 👉
1565
+ >
1566
+ > Environment variables and tilde (~) not allowed here. `dar` does not expand them.
1567
+ >
1568
+ > See [TODO](#todo)
1569
+
1536
1570
  ```text
1537
1571
  # Switch to ordered selection mode, which means that the following
1538
1572
  # options will be considered top to bottom
@@ -1906,6 +1940,8 @@ These [.darrc](#darrc) settings make `dar` print the current directory being pro
1906
1940
 
1907
1941
  This is very useful in very long running jobs to get an indication that the backup is proceeding normally.
1908
1942
 
1943
+ The `dar` output is streamed to the `dar-backup-commands.log` file.
1944
+
1909
1945
  ### Separate log file for command output
1910
1946
 
1911
1947
  Dar-backup's log file is called `dar-backup.log`.
@@ -2067,8 +2103,54 @@ Then reload Zsh:
2067
2103
  source ~/.zshrc
2068
2104
  ```
2069
2105
 
2106
+ ## Easy development setup
2107
+
2108
+ It is very easy to have your own development environment.
2109
+
2110
+ ```bash
2111
+ git clone https://github.com/per2jensen/dar-backup.git
2112
+ cd dar-backup/v2
2113
+ ./build.py
2114
+ ```
2115
+
2116
+ This script:
2117
+
2118
+ - Creates a Python virtual environment called `venv`
2119
+ - pip install `hatch`
2120
+ - pip install the development environment as setup in pyproject.toml
2121
+
2122
+ --
2123
+
2124
+ ```text
2125
+ dev = [
2126
+ "pytest",
2127
+ "wheel>=0.45.1",
2128
+ "requests>=2.32.2",
2129
+ "coverage>=7.8.2",
2130
+ "pytest>=8.4.0",
2131
+ "pytest-cov>=6.1.1",
2132
+ "psutil>=7.0.0",
2133
+ "pytest-timeout>=2.4.0",
2134
+ "httpcore>=0.17.3",
2135
+ "h11>=0.16.0",
2136
+ "zipp>=3.19.1",
2137
+ "anyio>=4.4.0",
2138
+ "black>=25.1.0"]
2139
+ ```
2140
+
2141
+ ✅ Your environment is now ready to activate and test!
2142
+
2143
+ Activate and run the test suite:
2144
+
2145
+ ```bash
2146
+ source venv/bin/activate # activate the virtual env
2147
+ pytest # run the test suite
2148
+ ```
2149
+
2070
2150
  ## Todo
2071
2151
 
2152
+ - Perhaps look into pre-processing backup definitions. As `dar` does not expand env vars
2153
+ `dar-backup` could do so and feed the result to `dar`.
2072
2154
  - When run interactively, a progress bar during test and par2 generation would be nice.
2073
2155
  - Look into a way to move the .par2 files away from the `dar` slices, to maximize chance of good redundancy.
2074
2156
  - Add option to dar-backup to use the `dar` option `--fsa-scope none`
@@ -2233,12 +2315,12 @@ Sets up `dar-backup` according to provided config file.
2233
2315
  The installer creates the necessary backup catalog databases if `--create-db` is given.
2234
2316
 
2235
2317
  ```bash
2236
- --config Sets up `dar-backup`.
2237
- --create-db Create backup catalog databases. Add it to --config
2238
- --install-autocompletion Add bash or zsh auto completion - idempotent
2239
- --remove-autocompletion Remove the auto completion from bash or zsh
2318
+ --config Path to a config file. The configured directories will be created.
2319
+ --create-db Create backup catalog databases. Use this option with `--config`.
2320
+ --install-autocompletion Add bash or zsh auto completion - idempotent.
2321
+ --remove-autocompletion Remove the auto completion from bash or zsh.
2240
2322
  -v, --version Display version and licensing information.
2241
- -h, --help Displays usage info
2323
+ -h, --help Displays usage info.
2242
2324
  ```
2243
2325
 
2244
2326
  ### Demo options
@@ -2258,7 +2340,7 @@ Create directories:
2258
2340
  Sets up demo config files:
2259
2341
 
2260
2342
  - ~/.config/dar-backup/dar-backup.conf
2261
- - ~/.config/dar-backup/backup.d/default
2343
+ - ~/.config/dar-backup/backup.d/demo
2262
2344
 
2263
2345
  ```bash
2264
2346
  -i, --install Sets up `dar-backup`.
@@ -0,0 +1,25 @@
1
+ dar_backup/.darrc,sha256=-aerqivZmOsW_XBCh9IfbYTUvw0GkzDSr3Vx4GcNB1g,2113
2
+ dar_backup/Changelog.md,sha256=CpUyWEnzVvn2otCGTp-N4R_-0zlr1kFe_Y0yQ-xQEZg,11872
3
+ dar_backup/README.md,sha256=HWrcpd_nhzhL2quLlxp7Xd0i05GxZvJMu5nSw8lJKLk,59901
4
+ dar_backup/__about__.py,sha256=4n6dyFqIrCJMAEZWvi2IbAzYCvkz4sxqN2wbVMbVWNk,344
5
+ dar_backup/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
6
+ dar_backup/clean_log.py,sha256=pmmyPmLWbm3_3sHwJt9V_xBwUF8v015iS17ypJAGAZ4,6023
7
+ dar_backup/cleanup.py,sha256=_ggDcpMCB1MhXStvYussp_PdGfhIFtEutT5BNrNkMSY,13297
8
+ dar_backup/command_runner.py,sha256=IUPYYBsaDFBp0q81Rt2xB9ucuK9eu5bXziRKXhI5YZM,4500
9
+ dar_backup/config_settings.py,sha256=2UAHvatrVO4ark6lCn2Q7qBvZN6DUMK2ftlNrKpzlWc,5792
10
+ dar_backup/dar-backup.conf,sha256=46V2zdvjj_aThFY11wWlffjmoiChYmatdf5DXCsmmao,1208
11
+ dar_backup/dar-backup.conf.j2,sha256=z3epGo6nB_Jh3liTOp93wJO_HKUsf7qghe9cdtFH7cY,2021
12
+ dar_backup/dar_backup.py,sha256=pgUQ1wZ5_yocRJJDRa4AkVyI7I8Z5WBY5vD5a4ASkmA,43001
13
+ dar_backup/dar_backup_systemd.py,sha256=PwAc2H2J3hQLWpnC6Ib95NZYtB2G2NDgkSblfLj1n10,3875
14
+ dar_backup/demo.py,sha256=bxEq_nJwHuQydERPppkvhotg1fdwBX_CE33m5fX_kxw,7945
15
+ dar_backup/demo_backup_def.j2,sha256=hQW2Glp0QGV3Kt8cwjS0mpOCdyzjVlpgbgL6LpXTKJA,1793
16
+ dar_backup/exceptions.py,sha256=dBQwgKUilgfb1Tnlm_m1Dl1IqoY75L0n1uRkm0DBVTk,158
17
+ dar_backup/installer.py,sha256=xSXh77qquIZbUTSY3AbhERQbS7bnrPE__M_yqpszdhM,6883
18
+ dar_backup/manager.py,sha256=d1zliFpSuWc8JhjKqLMC-xOhp5kSIcfeGkMZOVuZcM0,27143
19
+ dar_backup/rich_progress.py,sha256=SfwFxebBl6jnDQMUQr4McknkW1yQWaJVo1Ju1OD3okA,3221
20
+ dar_backup/util.py,sha256=iTOGsZyIdkvh9tIu7hD_IXi-9HO6GhVgqact5GGInEY,26063
21
+ dar_backup-0.8.0.dist-info/METADATA,sha256=_CBNbG6I0C8C18V3ZfZPGWuMSnVD0UrXlswQCK-OnCY,102592
22
+ dar_backup-0.8.0.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
23
+ dar_backup-0.8.0.dist-info/entry_points.txt,sha256=pOK9M8cHeAcGIatrYzkm_1O89kPk0enyYONALYjFBx4,286
24
+ dar_backup-0.8.0.dist-info/licenses/LICENSE,sha256=OXLcl0T2SZ8Pmy2_dmlvKuetivmyPd5m1q-Gyd-zaYY,35149
25
+ dar_backup-0.8.0.dist-info/RECORD,,
@@ -1,25 +0,0 @@
1
- dar_backup/.darrc,sha256=-aerqivZmOsW_XBCh9IfbYTUvw0GkzDSr3Vx4GcNB1g,2113
2
- dar_backup/Changelog.md,sha256=sf00ukQPTTqwraUf-NaGtMlWRBjgsQ2wyoEkF1rBew8,10834
3
- dar_backup/README.md,sha256=Sc1721Ar5PVj2tkhjrmyJXt3a5iCAlLSRjyYvUi8S9s,58463
4
- dar_backup/__about__.py,sha256=EL4Ktx4eIjaIDyiDjA8mHNzo4u6QvMkXc0WXCOClYhg,344
5
- dar_backup/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
6
- dar_backup/clean_log.py,sha256=tinRODoWWKvs6Ct41j2E4K6Ktw8CeZlsQVtyYioVVfQ,5492
7
- dar_backup/cleanup.py,sha256=VJdFHch21GWWGv1b_GlVK6dtxuUkHUazTVwmt-O3Gds,12691
8
- dar_backup/command_runner.py,sha256=IUPYYBsaDFBp0q81Rt2xB9ucuK9eu5bXziRKXhI5YZM,4500
9
- dar_backup/config_settings.py,sha256=1iRcurEPJObzVaYtiVRhXJi5DFT04VuLKbF9WIHbCvw,5305
10
- dar_backup/dar-backup.conf,sha256=46V2zdvjj_aThFY11wWlffjmoiChYmatdf5DXCsmmao,1208
11
- dar_backup/dar-backup.conf.j2,sha256=QdFXcuLFbz3UOOqR5SuHCJqds5cmtVnWrxW5Eg3sFwU,1871
12
- dar_backup/dar_backup.py,sha256=6bcFRHyMoy1epxQ1AqKnNk2L72wx9WifNuMZf0oFi2I,42586
13
- dar_backup/dar_backup_systemd.py,sha256=PwAc2H2J3hQLWpnC6Ib95NZYtB2G2NDgkSblfLj1n10,3875
14
- dar_backup/demo.py,sha256=bxEq_nJwHuQydERPppkvhotg1fdwBX_CE33m5fX_kxw,7945
15
- dar_backup/demo_backup_def.j2,sha256=hQW2Glp0QGV3Kt8cwjS0mpOCdyzjVlpgbgL6LpXTKJA,1793
16
- dar_backup/exceptions.py,sha256=dBQwgKUilgfb1Tnlm_m1Dl1IqoY75L0n1uRkm0DBVTk,158
17
- dar_backup/installer.py,sha256=bLDlJnGNExHElr7xp-G5o9IC9pN4jmhaPdphfK4VoAE,6492
18
- dar_backup/manager.py,sha256=ygBgxMHGRWtbSxylWsK3AugsQamWL68BAsWPjDpUEvo,26766
19
- dar_backup/rich_progress.py,sha256=rfKcE9P7hIGmmz9E1zMqC98axZk9fq1o-lVPeMvH2MI,3173
20
- dar_backup/util.py,sha256=_7RLLKsOoGktAgajjSYL29rs4jBJzRw0F21FXXLBnDE,24154
21
- dar_backup-0.7.1.dist-info/METADATA,sha256=sNYO2Vx8AIrwhxtISFXHEBpNpviui65IIlHo8hv1fI0,100213
22
- dar_backup-0.7.1.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
23
- dar_backup-0.7.1.dist-info/entry_points.txt,sha256=pOK9M8cHeAcGIatrYzkm_1O89kPk0enyYONALYjFBx4,286
24
- dar_backup-0.7.1.dist-info/licenses/LICENSE,sha256=OXLcl0T2SZ8Pmy2_dmlvKuetivmyPd5m1q-Gyd-zaYY,35149
25
- dar_backup-0.7.1.dist-info/RECORD,,