dar-backup 0.7.1__py3-none-any.whl → 0.7.2__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
dar_backup/Changelog.md CHANGED
@@ -1,7 +1,24 @@
1
1
  <!-- markdownlint-disable MD024 -->
2
2
  # dar-backup Changelog
3
3
 
4
- ## v2-beta-0.6.21 - not yet released
4
+ ## v2-beta-0.7.2 - 2025-06-07
5
+
6
+ Github link: [v2-beta-0.7.2](https://github.com/per2jensen/dar-backup/tree/v2-beta-0.7.2/v2)
7
+
8
+ ### Added
9
+
10
+ - Refactored build system, so all dependencies are kept in `pyproject.toml`. The dependencies are separated into dev, packaging and delivery phases.
11
+ - Use `build.sh` to setup pytest environment in Github workflow.
12
+
13
+ - Do the same to get a development environment going.
14
+
15
+ - Added 2 new optional params to control log file roll.
16
+ - Enrolling into [Snyk code checker](https://snyk.io/code-checker/) and learning how to work with it.
17
+
18
+ - Snyk helped pointing out vulnerable versions of some packages used.
19
+ - Input sanitization started, there is room for improvement.
20
+
21
+ ## v2-beta-0.7.1 - 2025-05-22
5
22
 
6
23
  Github link: [v2-beta-0.7.1](https://github.com/per2jensen/dar-backup/tree/v2-beta-0.7.1/v2)
7
24
 
@@ -24,9 +41,9 @@ Github link: [v2-beta-0.7.1](https://github.com/per2jensen/dar-backup/tree/v2-be
24
41
  - Action + program to capture cloning stats and store them in v2/doc directory. Includes a badge.
25
42
 
26
43
  -- annotate new daily max number of clones
27
- -- Celebration badge when number of clones numbers are hit (just for fun)
44
+ -- Celebration badge when certain clones numbers are hit (just for fun)
28
45
 
29
- - Action + program to generate 12weeks cloning dashboard (a PNG) with annotation
46
+ - Action + program to generate 12 weeks cloning dashboard (a PNG) with annotation
30
47
  - Tweaked the auto completion setup in .bashrc, it stopped working for me unknown reasons (needs some looking into)
31
48
  - --verbose now affects the startup banner. Now it is printed only if --verbose is given
32
49
 
dar_backup/README.md CHANGED
@@ -3,15 +3,16 @@
3
3
 
4
4
  **Reliable DAR backups, automated in clean Python**
5
5
 
6
- [![codecov](https://codecov.io/gh/per2jensen/dar-backup/branch/main/graph/badge.svg)](https://codecov.io/gh/per2jensen/dar-backup)
6
+ [![Codecov](https://codecov.io/gh/per2jensen/dar-backup/branch/main/graph/badge.svg)](https://codecov.io/gh/per2jensen/dar-backup)
7
+ [![Snyk Vuln findings](https://snyk.io/test/github/per2jensen/dar-backup/badge.svg)](https://snyk.io/test/github/per2jensen/dar-backup)
7
8
  ![CI](https://github.com/per2jensen/dar-backup/actions/workflows/py-tests.yml/badge.svg)
8
9
  [![PyPI version](https://img.shields.io/pypi/v/dar-backup.svg)](https://pypi.org/project/dar-backup/)
9
- [![PyPI total Downloads](https://img.shields.io/badge/dynamic/json?color=blue&label=Total%20Downloads&query=total&url=https%3A%2F%2Fraw.githubusercontent.com%2Fper2jensen%2Fdar-backup%2Fmain%2Fdownloads.json)](https://pypi.org/project/dar-backup/)
10
+ [![PyPI downloads](https://img.shields.io/badge/dynamic/json?color=blue&label=PyPI%20downloads&query=total&url=https%3A%2F%2Fraw.githubusercontent.com%2Fper2jensen%2Fdar-backup%2Fmain%2Fdownloads.json)](https://pypi.org/project/dar-backup/)
10
11
  [![# clones](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/per2jensen/dar-backup/main/v2/doc/badges/badge_clones.json)](https://github.com/per2jensen/dar-backup/blob/main/v2/doc/weekly_clones.png)
11
12
  [![Milestone](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/per2jensen/dar-backup/main/v2/doc/badges/milestone_badge.json)](https://github.com/per2jensen/dar-backup/blob/main/v2/doc/weekly_clones.png)
12
13
 
13
14
  The wonderful 'dar' [Disk Archiver](https://github.com/Edrusb/DAR) is used for
14
- the heavy lifting, together with [parchive](https://github.com/Parchive/par2cmdline) suite in these scripts.
15
+ the heavy lifting, together with the [parchive](https://github.com/Parchive/par2cmdline) suite in these scripts.
15
16
 
16
17
  This is the `Python` based [**version 2**](https://github.com/per2jensen/dar-backup/tree/main/v2) of `dar-backup`.
17
18
 
@@ -73,6 +74,7 @@ This is the `Python` based [**version 2**](https://github.com/per2jensen/dar-bac
73
74
  - [Skipping cache directories](#skipping-cache-directories)
74
75
  - [Progress bar + current directory](#progress-bar-and-current-directory)
75
76
  - [Shell Autocompletion](#shell-autocompletion)
77
+ - [Easy development setup](#easy-development-setup)
76
78
  - [Todo](#todo)
77
79
  - [Known Limitations / Edge Cases](#known-limitations--edge-cases)
78
80
  - [Reference](#reference)
@@ -207,17 +209,17 @@ Setup the demo configurations and show a few operations
207
209
  <br>
208
210
 
209
211
  ```bash
210
- # see reference section for options tweaking the install
212
+ # See reference section for options tweaking the install
211
213
  demo --install
212
214
 
213
215
  # create catalog database
214
216
  manager --create-db
215
217
 
216
- # do a FULL backup of the installed backup definition called `demo`
218
+ # FULL backup as defined in backup definition `demo`
217
219
  dar-backup --full-backup
218
220
 
219
- # list the contents of the backup
220
- dar-backup --list-contents $(dar-backup --list |tail -n 1 | cut -d " " -f1)
221
+ # List the contents of the backup
222
+ dar-backup --list-contents demo_FULL_$(date '+%F')
221
223
  ```
222
224
 
223
225
  <details>
@@ -259,7 +261,7 @@ Config file: /home/user/.config/dar-backup/dar-backup.conf
259
261
 
260
262
 
261
263
 
262
- (venv) $ dar-backup --list-contents $(dar-backup --list |tail -n 1 | cut -d " " -f1)
264
+ (venv) $ dar-backup --list-contents demo_FULL_$(date '+%F')
263
265
  ========== Startup Settings ==========
264
266
  dar-backup.py: 0.7.1
265
267
  dar path: /home/user/.local/dar/bin/dar
@@ -287,10 +289,10 @@ Config file: /home/user/.config/dar-backup/dar-backup.conf
287
289
  Perform a restore and show the restored files
288
290
 
289
291
  ```bash
290
- # restore all files in the backup
291
- dar-backup --restore $(dar-backup --list |tail -n 1 | cut -d " " -f1) --verbose
292
+ # Restore all files in the backup
293
+ dar-backup --restore demo_FULL_$(date '+%F') --verbose
292
294
 
293
- # prove the files have been restored to directory as configured
295
+ # Prove the files have been restored to directory as configured
294
296
  find $HOME/dar-backup/restore
295
297
  ```
296
298
 
@@ -299,7 +301,7 @@ find $HOME/dar-backup/restore
299
301
  <summary>🎯 --restore details</summary>
300
302
 
301
303
  ```bash
302
- (venv) $ dar-backup --verbose --restore $(dar-backup --list |tail -n 1 | cut -d " " -f1)
304
+ (venv) $ dar-backup --restore demo_FULL_$(date '+%F') --verbose
303
305
  ========== Startup Settings ==========
304
306
  dar-backup.py: 0.7.1
305
307
  dar path: /home/user/.local/dar/bin/dar
@@ -337,7 +339,7 @@ PAR2 enabled: True
337
339
 
338
340
  > ✅ **Next steps**
339
341
  >
340
- > Tinker with `demo's` options:
342
+ > Play with `demo's` options:
341
343
  >
342
344
  > - --root-dir (perhaps $HOME)
343
345
  > - --dir-to-backup (perhaps Pictures)
@@ -472,8 +474,9 @@ Please review the [Code of Conduct](https://github.com/per2jensen/dar-backup/blo
472
474
 
473
475
  ## Requirements
474
476
 
477
+ - A linux system
475
478
  - dar
476
- - par2
479
+ - parchive (par2)
477
480
  - python3
478
481
  - python3-venv
479
482
 
@@ -578,27 +581,25 @@ See section 15 and section 16 in the supplied "LICENSE" file.
578
581
 
579
582
  ### 2 - configuration
580
583
 
581
- The dar-backup [demo](#demo-options) application can be used to demo how `dar-backup` works.
582
- It creates some directories, installs a demo configuration file and puts a demo backup definition in place.
584
+ The dar-backup [installer](#installer-options) application can be used to setup the needed directories for `dar-backup` to work.
585
+ It creates necessary directories as prescribed in the config file and optionally creates manager databases.
583
586
 
584
- `demo` is non-destructive and stops if some of the default directories exist.
587
+ `installer` can also add configuration of shell auto completion.
585
588
 
586
- Run `demo`
589
+ Step 1:
587
590
 
588
- ```bash
589
- demo --install
590
- ```
591
+ Create a config file - [see details on config file](#config-file))
591
592
 
592
- The output is
593
+ Step 2:
593
594
 
594
- ```text
595
- Directories created: `/home/user/dar-backup/` and `/home/user/.config/dar-backup`
596
- Config file deployed to /home/user/.config/dar-backup/dar-backup.conf
597
- Default backup definition deployed to /home/user/.config/dar-backup/backup.d/default
598
- 1. Now run `manager --create-db` to create the catalog database.
599
- 2. Then you can run `dar-backup --full-backup` to create a backup.
600
- 3. List backups with `dar-backup --list`
601
- 4. List contents of a backup with `dar-backup --list-contents <backup-name>`
595
+ Create one or more backup definitions - [see details on backup definitions](#backup-definition-example)
596
+
597
+ Step 3:
598
+
599
+ Run the installer:
600
+
601
+ ```bash
602
+ installer --config <path to dar-backup.conf> --install-autocompletion
602
603
  ```
603
604
 
604
605
  ### 3 - generate catalog databases
@@ -613,9 +614,7 @@ manager --create-db
613
614
 
614
615
  ### 4 - give dar-backup a spin
615
616
 
616
- The `demo` application has put a demo [backup definition](#backup-definition-example) in place in BACKUP.D_DIR (see [config file](#config-file)).
617
-
618
- You are now ready to do backups as configured in the backup definition.
617
+ You are now ready to do backups as configured in your backup definition(s).
619
618
 
620
619
  Give `dar-backup`a spin:
621
620
 
@@ -625,6 +624,9 @@ dar-backup --full-backup --verbose
625
624
  # list backups
626
625
  dar-backup --list
627
626
 
627
+ # list contents of a dar backup
628
+ dar-backup --list-contents <TAB>... <choose a backup>
629
+
628
630
  # see some examples on usage
629
631
  dar-backup --examples
630
632
 
@@ -663,6 +665,9 @@ Tilde `~` and environment variables can be used in the paths for various file lo
663
665
  ```text
664
666
  [MISC]
665
667
  LOGFILE_LOCATION=~/.dar-backup.log
668
+ # optional parameters
669
+ # LOGFILE_MAX_BYTES = 26214400 # 25 MB max file size is default, change as neeeded
670
+ # LOGFILE_BACKUP_COUNT = 5 # 5 backup log files is default, change as needed
666
671
  MAX_SIZE_VERIFICATION_MB = 20
667
672
  MIN_SIZE_VERIFICATION_MB = 1
668
673
  NO_FILES_VERIFICATION = 5
@@ -681,6 +686,8 @@ TEST_RESTORE_DIR = /tmp/dar-backup/restore/
681
686
 
682
687
  [AGE]
683
688
  # age settings are in days
689
+ # `cleanup` script removes archives and their .par redundancy files if older than configured.
690
+ # `cleanup` does not remove FULL archives, unless specifically told to and a "y" is answered to "are you sure?".
684
691
  DIFF_AGE = 100
685
692
  INCR_AGE = 40
686
693
 
@@ -836,6 +843,12 @@ The name of the file is the name of the backup definition.
836
843
 
837
844
  You can use as many backup definitions as you need.
838
845
 
846
+ > Note 👉
847
+ >
848
+ > Environment variables and tilde (~) not allowed here. `dar` does not expand them.
849
+ >
850
+ > See [TODO](#todo)
851
+
839
852
  ```text
840
853
  # Switch to ordered selection mode, which means that the following
841
854
  # options will be considered top to bottom
@@ -1209,6 +1222,8 @@ These [.darrc](#darrc) settings make `dar` print the current directory being pro
1209
1222
 
1210
1223
  This is very useful in very long running jobs to get an indication that the backup is proceeding normally.
1211
1224
 
1225
+ The `dar` output is streamed to the `dar-backup-commands.log` file.
1226
+
1212
1227
  ### Separate log file for command output
1213
1228
 
1214
1229
  Dar-backup's log file is called `dar-backup.log`.
@@ -1370,8 +1385,54 @@ Then reload Zsh:
1370
1385
  source ~/.zshrc
1371
1386
  ```
1372
1387
 
1388
+ ## Easy development setup
1389
+
1390
+ It is very easy to have your own development environment.
1391
+
1392
+ ```bash
1393
+ git clone https://github.com/per2jensen/dar-backup.git
1394
+ cd dar-backup/v2
1395
+ ./build.py
1396
+ ```
1397
+
1398
+ This script:
1399
+
1400
+ - Creates a Python virtual environment called `venv`
1401
+ - pip install `hatch`
1402
+ - pip install the development environment as setup in pyproject.toml
1403
+
1404
+ --
1405
+
1406
+ ```text
1407
+ dev = [
1408
+ "pytest",
1409
+ "wheel>=0.45.1",
1410
+ "requests>=2.32.2",
1411
+ "coverage>=7.8.2",
1412
+ "pytest>=8.4.0",
1413
+ "pytest-cov>=6.1.1",
1414
+ "psutil>=7.0.0",
1415
+ "pytest-timeout>=2.4.0",
1416
+ "httpcore>=0.17.3",
1417
+ "h11>=0.16.0",
1418
+ "zipp>=3.19.1",
1419
+ "anyio>=4.4.0",
1420
+ "black>=25.1.0"]
1421
+ ```
1422
+
1423
+ ✅ Your environment is now ready to activate and test!
1424
+
1425
+ Activate and run the test suite:
1426
+
1427
+ ```bash
1428
+ source venv/bin/activate # activate the virtual env
1429
+ pytest # run the test suite
1430
+ ```
1431
+
1373
1432
  ## Todo
1374
1433
 
1434
+ - Perhaps look into pre-processing backup definitions. As `dar` does not expand env vars
1435
+ `dar-backup` could do so and feed the result to `dar`.
1375
1436
  - When run interactively, a progress bar during test and par2 generation would be nice.
1376
1437
  - Look into a way to move the .par2 files away from the `dar` slices, to maximize chance of good redundancy.
1377
1438
  - Add option to dar-backup to use the `dar` option `--fsa-scope none`
@@ -1536,12 +1597,12 @@ Sets up `dar-backup` according to provided config file.
1536
1597
  The installer creates the necessary backup catalog databases if `--create-db` is given.
1537
1598
 
1538
1599
  ```bash
1539
- --config Sets up `dar-backup`.
1540
- --create-db Create backup catalog databases. Add it to --config
1541
- --install-autocompletion Add bash or zsh auto completion - idempotent
1542
- --remove-autocompletion Remove the auto completion from bash or zsh
1600
+ --config Path to a config file. The configured directories will be created.
1601
+ --create-db Create backup catalog databases. Use this option with `--config`.
1602
+ --install-autocompletion Add bash or zsh auto completion - idempotent.
1603
+ --remove-autocompletion Remove the auto completion from bash or zsh.
1543
1604
  -v, --version Display version and licensing information.
1544
- -h, --help Displays usage info
1605
+ -h, --help Displays usage info.
1545
1606
  ```
1546
1607
 
1547
1608
  ### Demo options
@@ -1561,7 +1622,7 @@ Create directories:
1561
1622
  Sets up demo config files:
1562
1623
 
1563
1624
  - ~/.config/dar-backup/dar-backup.conf
1564
- - ~/.config/dar-backup/backup.d/default
1625
+ - ~/.config/dar-backup/backup.d/demo
1565
1626
 
1566
1627
  ```bash
1567
1628
  -i, --install Sets up `dar-backup`.
dar_backup/__about__.py CHANGED
@@ -1,4 +1,4 @@
1
- __version__ = "0.7.1"
1
+ __version__ = "0.7.2"
2
2
 
3
3
  __license__ = '''Licensed under GNU GENERAL PUBLIC LICENSE v3, see the supplied file "LICENSE" for details.
4
4
  THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW, not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
dar_backup/clean_log.py CHANGED
@@ -137,6 +137,7 @@ def main():
137
137
  args.file = [config_settings.logfile_location]
138
138
 
139
139
  for file_path in args.file:
140
+
140
141
  if not isinstance(file_path, (str, bytes, os.PathLike)):
141
142
  print(f"Error: Invalid file path type: {file_path}")
142
143
  sys.exit(1)
dar_backup/cleanup.py CHANGED
@@ -41,6 +41,8 @@ from dar_backup.util import show_version
41
41
  from dar_backup.util import get_invocation_command_line
42
42
  from dar_backup.util import print_aligned_settings
43
43
  from dar_backup.util import backup_definition_completer, list_archive_completer
44
+ from dar_backup.util import is_safe_filename
45
+ from dar_backup.util import show_scriptname
44
46
 
45
47
  from dar_backup.command_runner import CommandRunner
46
48
  from dar_backup.command_runner import CommandResult
@@ -80,7 +82,7 @@ def delete_old_backups(backup_dir, age, backup_type, args, backup_definition=Non
80
82
  if file_date < cutoff_date:
81
83
  file_path = os.path.join(backup_dir, filename)
82
84
  try:
83
- os.remove(file_path)
85
+ is_safe_filename(file_path) and os.remove(file_path)
84
86
  logger.info(f"Deleted {backup_type} backup: {file_path}")
85
87
  archive_name = filename.split('.')[0]
86
88
  if not archive_name in archives_deleted:
@@ -109,7 +111,7 @@ def delete_archive(backup_dir, archive_name, args):
109
111
  if archive_regex.match(filename):
110
112
  file_path = os.path.join(backup_dir, filename)
111
113
  try:
112
- os.remove(file_path)
114
+ is_safe_filename(file_path) and os.remove(file_path)
113
115
  logger.info(f"Deleted archive slice: {file_path}")
114
116
  files_deleted = True
115
117
  except Exception as e:
@@ -127,7 +129,7 @@ def delete_archive(backup_dir, archive_name, args):
127
129
  if par2_regex.match(filename):
128
130
  file_path = os.path.join(backup_dir, filename)
129
131
  try:
130
- os.remove(file_path)
132
+ is_safe_filename(file_path) and os.remove(file_path)
131
133
  logger.info(f"Deleted PAR2 file: {file_path}")
132
134
  files_deleted = True
133
135
  except Exception as e:
@@ -215,13 +217,13 @@ def main():
215
217
 
216
218
  # command_output_log = os.path.join(config_settings.logfile_location.removesuffix("dar-backup.log"), "dar-backup-commands.log")
217
219
  command_output_log = config_settings.logfile_location.replace("dar-backup.log", "dar-backup-commands.log")
218
- logger = setup_logging(config_settings.logfile_location, command_output_log, args.log_level, args.log_stdout)
220
+ logger = setup_logging(config_settings.logfile_location, command_output_log, args.log_level, args.log_stdout, logfile_max_bytes=config_settings.logfile_max_bytes, logfile_backup_count=config_settings.logfile_backup_count)
219
221
  command_logger = get_logger(command_output_logger = True)
220
222
  runner = CommandRunner(logger=logger, command_logger=command_logger)
221
223
 
222
224
  start_msgs: List[Tuple[str, str]] = []
223
225
 
224
- start_msgs.append(("cleanup.py:", about.__version__))
226
+ start_msgs.append((f"{show_scriptname()}:", about.__version__))
225
227
 
226
228
  logger.info(f"START TIME: {start_time}")
227
229
  logger.debug(f"Command line: {get_invocation_command_line()}")
@@ -233,6 +235,8 @@ def main():
233
235
  start_msgs.append(("Config file:", args.config_file))
234
236
  args.verbose and start_msgs.append(("Backup dir:", config_settings.backup_dir))
235
237
  start_msgs.append(("Logfile:", config_settings.logfile_location))
238
+ args.verbose and start_msgs.append(("Logfile max size (bytes):", config_settings.logfile_max_bytes))
239
+ args.verbose and start_msgs.append(("Logfile backup count:", config_settings.logfile_backup_count))
236
240
  args.verbose and start_msgs.append(("--alternate-archive-dir:", args.alternate_archive_dir))
237
241
  args.verbose and start_msgs.append(("--cleanup-specific-archives:", args.cleanup_specific_archives))
238
242
 
@@ -261,7 +265,8 @@ def main():
261
265
  if "_FULL_" in archive_name:
262
266
  if not confirm_full_archive_deletion(archive_name, args.test_mode):
263
267
  continue
264
- logger.info(f"Deleting archive: {archive_name}")
268
+ archive_path = os.path.join(config_settings.backup_dir, archive_name.strip())
269
+ logger.info(f"Deleting archive: {archive_path}")
265
270
  delete_archive(config_settings.backup_dir, archive_name.strip(), args)
266
271
  elif args.list:
267
272
  list_backups(config_settings.backup_dir, args.backup_definition)
@@ -44,6 +44,8 @@ class ConfigSettings:
44
44
  incr_age: int = field(init=False)
45
45
  error_correction_percent: int = field(init=False)
46
46
  par2_enabled: bool = field(init=False)
47
+ logfile_max_bytes: int = field(init=False)
48
+ logfile_no_count: int = field(init=False)
47
49
 
48
50
 
49
51
  OPTIONAL_CONFIG_FIELDS = [
@@ -54,6 +56,20 @@ class ConfigSettings:
54
56
  "type": str,
55
57
  "default": None,
56
58
  },
59
+ {
60
+ "section": "MISC",
61
+ "key": "LOGFILE_MAX_BYTES",
62
+ "attr": "logfile_max_bytes",
63
+ "type": int,
64
+ "default": 26214400 , # 25 MB
65
+ },
66
+ {
67
+ "section": "MISC",
68
+ "key": "LOGFILE_BACKUP_COUNT",
69
+ "attr": "logfile_backup_count",
70
+ "type": int,
71
+ "default": 5,
72
+ },
57
73
  # Add more optional fields here
58
74
  ]
59
75
 
@@ -26,6 +26,10 @@
26
26
 
27
27
  [MISC]
28
28
  LOGFILE_LOCATION = {{ vars_map.DAR_BACKUP_DIR -}}/dar-backup.log
29
+ # optional parameters
30
+ # LOGFILE_MAX_BYTES = 26214400 # 25 MB default, change as neeeded
31
+ # LOGFILE_BACKUP_COUNT = 5 # default, change as needed
32
+
29
33
  MAX_SIZE_VERIFICATION_MB = 2
30
34
  MIN_SIZE_VERIFICATION_MB = 0
31
35
  NO_FILES_VERIFICATION = 1
dar_backup/dar_backup.py CHANGED
@@ -55,6 +55,8 @@ from dar_backup.util import get_invocation_command_line
55
55
  from dar_backup.util import get_binary_info
56
56
  from dar_backup.util import print_aligned_settings
57
57
  from dar_backup.util import backup_definition_completer, list_archive_completer
58
+ from dar_backup.util import show_scriptname
59
+ from dar_backup.util import print_debug
58
60
 
59
61
  from dar_backup.command_runner import CommandRunner
60
62
  from dar_backup.command_runner import CommandResult
@@ -92,6 +94,7 @@ def generic_backup(type: str, command: List[str], backup_file: str, backup_defin
92
94
  Returns:
93
95
  List of tuples (<msg>, <exit_code>) of errors not considered critical enough for raising an exception
94
96
  """
97
+
95
98
  result: List[tuple] = []
96
99
 
97
100
  logger.info(f"===> Starting {type} backup for {backup_definition}")
@@ -105,7 +108,6 @@ def generic_backup(type: str, command: List[str], backup_file: str, backup_defin
105
108
  stop_event = Event()
106
109
  session_marker = f"=== START BACKUP SESSION: {int(time())} ==="
107
110
  get_logger(command_output_logger=True).info(session_marker)
108
-
109
111
  progress_thread = threading.Thread(
110
112
  target=show_log_driven_bar,
111
113
  args=(log_path, stop_event, session_marker),
@@ -534,12 +536,12 @@ def perform_backup(args: argparse.Namespace, config_settings: ConfigSettings, ba
534
536
 
535
537
  # Perform backup
536
538
  backup_result = generic_backup(backup_type, command, backup_file, backup_definition_path, args.darrc, config_settings, args)
539
+
537
540
  if not isinstance(backup_result, list) or not all(isinstance(i, tuple) and len(i) == 2 for i in backup_result):
538
541
  logger.error("Unexpected return format from generic_backup")
539
542
  backup_result = [("Unexpected return format from generic_backup", 1)]
540
543
 
541
544
  results.extend(backup_result)
542
-
543
545
  logger.info("Starting verification...")
544
546
  verify_result = verify(args, backup_file, backup_definition_path, config_settings)
545
547
  if verify_result:
@@ -838,7 +840,7 @@ def main():
838
840
  if command_output_log == config_settings.logfile_location:
839
841
  print(f"Error: logfile_location in {args.config_file} does not end at 'dar-backup.log', exiting", file=stderr)
840
842
 
841
- logger = setup_logging(config_settings.logfile_location, command_output_log, args.log_level, args.log_stdout)
843
+ logger = setup_logging(config_settings.logfile_location, command_output_log, args.log_level, args.log_stdout, logfile_max_bytes=config_settings.logfile_max_bytes, logfile_backup_count=config_settings.logfile_backup_count)
842
844
  command_logger = get_logger(command_output_logger = True)
843
845
  runner = CommandRunner(logger=logger, command_logger=command_logger)
844
846
 
@@ -863,11 +865,11 @@ def main():
863
865
  start_msgs: List[Tuple[str, str]] = []
864
866
 
865
867
  start_time=int(time())
866
- start_msgs.append(('dar-backup.py:', about.__version__))
868
+ start_msgs.append((f"{show_scriptname()}:", about.__version__))
867
869
  logger.info(f"START TIME: {start_time}")
868
- logger.debug(f"Command line: {get_invocation_command_line()}")
869
- logger.debug(f"{'`Args`:\n'}{args}")
870
- logger.debug(f"{'`Config_settings`:\n'}{config_settings}")
870
+ logger.debug(f"Command line:\n{get_invocation_command_line()}")
871
+ logger.debug(f"`Args`:\n{args}")
872
+ logger.debug(f"`Config_settings`:\n{config_settings}")
871
873
  dar_properties = get_binary_info(command='dar')
872
874
  start_msgs.append(('dar path:', dar_properties['path']))
873
875
  start_msgs.append(('dar version:', dar_properties['version']))
@@ -890,6 +892,9 @@ def main():
890
892
  args.verbose and start_msgs.append(("Restore dir:", restore_dir))
891
893
 
892
894
  args.verbose and start_msgs.append(("Logfile location:", config_settings.logfile_location))
895
+ args.verbose and start_msgs.append(("Logfile max size (bytes):", config_settings.logfile_max_bytes))
896
+ args.verbose and start_msgs.append(("Logfile backup count:", config_settings.logfile_backup_count))
897
+
893
898
  args.verbose and start_msgs.append(("PAR2 enabled:", config_settings.par2_enabled))
894
899
  args.verbose and start_msgs.append(("--do-not-compare:", args.do_not_compare))
895
900
 
@@ -928,6 +933,7 @@ def main():
928
933
 
929
934
  requirements('POSTREQ', config_settings)
930
935
 
936
+
931
937
  except Exception as e:
932
938
  logger.error("Exception details:", exc_info=True)
933
939
  results.append((repr(e), 1))
dar_backup/installer.py CHANGED
@@ -12,6 +12,7 @@ from dar_backup.manager import create_db
12
12
  # Always expand manager DB dir correctly, using helper function
13
13
  from dar_backup.manager import get_db_dir
14
14
  from dar_backup.util import expand_path
15
+ from dar_backup.util import is_safe_path
15
16
 
16
17
  def install_autocompletion():
17
18
  """Detect user shell, choose RC file, and idempotently append autocompletion."""
@@ -99,7 +100,7 @@ def uninstall_autocompletion() -> str:
99
100
 
100
101
 
101
102
 
102
- def run_installer(config_file: str, create_db_flag: bool, install_ac_flag: bool):
103
+ def run_installer(config_file: str, create_db_flag: bool):
103
104
  """
104
105
  Run the installation process for dar-backup using the given config file.
105
106
 
@@ -116,6 +117,8 @@ def run_installer(config_file: str, create_db_flag: bool, install_ac_flag: bool)
116
117
  config_file = os.path.expanduser(os.path.expandvars(config_file))
117
118
  config_settings = ConfigSettings(config_file)
118
119
 
120
+ print(f"Using config settings: {config_settings}")
121
+
119
122
  # Set up logging
120
123
  command_log = config_settings.logfile_location.replace("dar-backup.log", "dar-backup-commands.log")
121
124
  logger = setup_logging(
@@ -137,6 +140,9 @@ def run_installer(config_file: str, create_db_flag: bool, install_ac_flag: bool)
137
140
  }
138
141
 
139
142
  for name, dir_path in required_dirs.items():
143
+ if not is_safe_path(dir_path):
144
+ logger.error(f"Unsafe path detected: {dir_path} ({name})")
145
+ raise ValueError(f"Unsafe path detected: {dir_path} ({name})")
140
146
  expanded = Path(expand_path(dir_path))
141
147
  if not expanded.exists():
142
148
  logger.info(f"Creating directory: {expanded} ({name})")
@@ -149,12 +155,11 @@ def run_installer(config_file: str, create_db_flag: bool, install_ac_flag: bool)
149
155
  print(f"Creating catalog for: {backup_def}")
150
156
  result = create_db(backup_def, config_settings, logger, runner)
151
157
  if result == 0:
152
- print(f"✔️ Catalog created (or already existed): {backup_def}")
158
+ print(f"✔️ Catalog created (or already exist): {backup_def}")
153
159
  else:
154
160
  print(f"❌ Failed to create catalog: {backup_def}")
155
161
 
156
162
 
157
-
158
163
  def main():
159
164
  parser = argparse.ArgumentParser(description="dar-backup installer")
160
165
  parser.add_argument("--config", required=False, help="Path to config file")
@@ -177,12 +182,17 @@ def main():
177
182
 
178
183
 
179
184
  if args.config:
185
+ if not os.path.exists(args.config):
186
+ print(f"❌ Config file does not exist: {args.config}")
187
+ return
180
188
  run_installer(args.config, args.create_db)
181
- elif args.install_autocompletion:
189
+
190
+ if args.install_autocompletion:
182
191
  install_autocompletion()
183
192
  elif args.remove_autocompletion:
184
193
  uninstall_autocompletion()
185
194
 
195
+
186
196
 
187
197
  if __name__ == "__main__":
188
198
  main()
dar_backup/manager.py CHANGED
@@ -40,6 +40,7 @@ from dar_backup.util import get_binary_info
40
40
  from dar_backup.util import show_version
41
41
  from dar_backup.util import get_invocation_command_line
42
42
  from dar_backup.util import print_aligned_settings
43
+ from dar_backup.util import show_scriptname
43
44
 
44
45
  from dar_backup.command_runner import CommandRunner
45
46
  from dar_backup.command_runner import CommandResult
@@ -544,14 +545,15 @@ def main():
544
545
  return
545
546
 
546
547
  command_output_log = config_settings.logfile_location.replace("dar-backup.log", "dar-backup-commands.log")
547
- logger = setup_logging(config_settings.logfile_location, command_output_log, args.log_level, args.log_stdout)
548
+ logger = setup_logging(config_settings.logfile_location, command_output_log, args.log_level, args.log_stdout, logfile_max_bytes=config_settings.logfile_max_bytes, logfile_backup_count=config_settings.logfile_backup_count)
548
549
  command_logger = get_logger(command_output_logger=True)
549
550
  runner = CommandRunner(logger=logger, command_logger=command_logger)
550
551
 
551
552
  start_msgs: List[Tuple[str, str]] = []
552
553
 
553
554
  start_time = int(time())
554
- start_msgs.append((f"{SCRIPTNAME}:", about.__version__))
555
+
556
+ start_msgs.append((f"{show_scriptname()}:", about.__version__))
555
557
  logger.info(f"START TIME: {start_time}")
556
558
  logger.debug(f"Command line: {get_invocation_command_line()}")
557
559
  logger.debug(f"`args`:\n{args}")
@@ -559,6 +561,8 @@ def main():
559
561
  start_msgs.append(("Config file:", args.config_file))
560
562
  args.verbose and start_msgs.append(("Backup dir:", config_settings.backup_dir))
561
563
  start_msgs.append(("Logfile:", config_settings.logfile_location))
564
+ args.verbose and start_msgs.append(("Logfile max size (bytes):", config_settings.logfile_max_bytes))
565
+ args.verbose and start_msgs.append(("Logfile backup count:", config_settings.logfile_backup_count))
562
566
  args.verbose and start_msgs.append(("--alternate-archive-dir:", args.alternate_archive_dir))
563
567
  args.verbose and start_msgs.append(("--remove-specific-archive:", args.remove_specific_archive))
564
568
  dar_manager_properties = get_binary_info(command='dar_manager')
@@ -102,3 +102,4 @@ def show_log_driven_bar(log_path: str, stop_event: Event, session_marker: str, m
102
102
  if stop_event.is_set():
103
103
  break
104
104
 
105
+ # Rich prints a \n here, I will live with it
dar_backup/util.py CHANGED
@@ -12,7 +12,9 @@ See section 15 and section 16 in the supplied "LICENSE" file
12
12
  import typing
13
13
  import locale
14
14
  import configparser
15
+ import inspect
15
16
  import logging
17
+
16
18
  import os
17
19
  import re
18
20
  import subprocess
@@ -28,6 +30,7 @@ import dar_backup.__about__ as about
28
30
  from argcomplete.completers import ChoicesCompleter
29
31
  from datetime import datetime
30
32
  from dar_backup.config_settings import ConfigSettings
33
+ from logging.handlers import RotatingFileHandler
31
34
  from pathlib import Path
32
35
  from rich.console import Console
33
36
  from rich.text import Text
@@ -39,7 +42,16 @@ from typing import Tuple
39
42
  logger=None
40
43
  secondary_logger=None
41
44
 
42
- def setup_logging(log_file: str, command_output_log_file: str, log_level: str = "info", log_to_stdout: bool = False) -> logging.Logger:
45
+ #def setup_logging(log_file: str, command_output_log_file: str, log_level: str = "info", log_to_stdout: bool = False) -> logging.Logger:
46
+ def setup_logging(
47
+ log_file: str,
48
+ command_output_log_file: str,
49
+ log_level: str = "info",
50
+ log_to_stdout: bool = False,
51
+ logfile_max_bytes: int = 26214400,
52
+ logfile_backup_count: int = 5,
53
+ ) -> logging.Logger:
54
+
43
55
  """
44
56
  Sets up logging for the main program and a separate secondary logfile for command outputs.
45
57
 
@@ -48,9 +60,11 @@ def setup_logging(log_file: str, command_output_log_file: str, log_level: str =
48
60
  command_output_log_file (str): The path to the secondary log file for command outputs.
49
61
  log_level (str): The log level to use. Can be "info", "debug", or "trace". Defaults to "info".
50
62
  log_to_stdout (bool): If True, log messages will be printed to the console. Defaults to False.
63
+ logfile_max_bytes: max file size of a log file, defailt = 26214400.
64
+ logfile_backup_count: max numbers of logs files, default = 5.
51
65
 
52
66
  Returns:
53
- None
67
+ a RotatingFileHandler logger instance.
54
68
 
55
69
  Raises:
56
70
  Exception: If an error occurs during logging initialization
@@ -66,20 +80,34 @@ def setup_logging(log_file: str, command_output_log_file: str, log_level: str =
66
80
 
67
81
  logging.Logger.trace = trace
68
82
 
83
+ file_handler = RotatingFileHandler(
84
+ log_file,
85
+ maxBytes=logfile_max_bytes,
86
+ backupCount=logfile_backup_count,
87
+ encoding="utf-8",
88
+ )
89
+
90
+ command_handler = RotatingFileHandler(
91
+ command_output_log_file,
92
+ maxBytes=logfile_max_bytes,
93
+ backupCount=logfile_backup_count,
94
+ encoding="utf-8",
95
+ )
96
+
97
+ formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
98
+ file_handler.setFormatter(formatter)
99
+ command_handler.setFormatter(formatter)
100
+
101
+
69
102
  # Setup main logger
70
103
  logger = logging.getLogger("main_logger")
71
104
  logger.setLevel(logging.DEBUG if log_level == "debug" else TRACE_LEVEL_NUM if log_level == "trace" else logging.INFO)
72
- formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
73
- file_handler = logging.FileHandler(log_file)
74
- file_handler.setFormatter(formatter)
75
105
  logger.addHandler(file_handler)
76
106
 
77
107
  # Setup secondary logger for command outputs
78
108
  secondary_logger = logging.getLogger("command_output_logger")
79
109
  secondary_logger.setLevel(logging.DEBUG if log_level == "debug" else TRACE_LEVEL_NUM if log_level == "trace" else logging.INFO)
80
- sec_file_handler = logging.FileHandler(command_output_log_file)
81
- sec_file_handler.setFormatter(formatter)
82
- secondary_logger.addHandler(sec_file_handler)
110
+ secondary_logger.addHandler(command_handler)
83
111
 
84
112
  if log_to_stdout:
85
113
  stdout_handler = logging.StreamHandler(sys.stdout)
@@ -124,6 +152,15 @@ completer_logger = _setup_completer_logger()
124
152
  completer_logger.debug("Completer logger initialized.")
125
153
 
126
154
 
155
+ def print_debug(msg):
156
+ """
157
+ Print a debug message with the filename and line number of the caller.
158
+ """
159
+ frame = inspect.currentframe().f_back
160
+ print(f"[DEBUG] {frame.f_code.co_filename}:{frame.f_lineno} - {repr(msg)}")
161
+
162
+
163
+
127
164
  def get_invocation_command_line() -> str:
128
165
  """
129
166
  Safely retrieves the exact command line used to invoke the current Python process.
@@ -146,6 +183,17 @@ def get_invocation_command_line() -> str:
146
183
  return f"[error: could not read /proc/[pid]/cmdline: {e}]"
147
184
 
148
185
 
186
+ def show_scriptname() -> str:
187
+ """
188
+ Return script name, useful in start banner for example
189
+ """
190
+ try:
191
+ scriptname = os.path.basename(sys.argv[0])
192
+ except:
193
+ scriptname = "unknown"
194
+ return scriptname
195
+
196
+
149
197
  def show_version():
150
198
  script_name = os.path.basename(sys.argv[0])
151
199
  print(f"{script_name} {about.__version__}")
@@ -683,3 +731,30 @@ def normalize_dir(path: str) -> str:
683
731
  normalized = str(p)
684
732
  return normalized
685
733
 
734
+
735
+
736
+ # Reusable pattern for archive file naming
737
+ archive_pattern = re.compile(
738
+ r'^.+?_(FULL|DIFF|INCR)_(\d{4}-\d{2}-\d{2})\.\d+\.dar'
739
+ r'(?:\.vol\d+(?:\+\d+)?\.par2|\.par2)?$'
740
+ )
741
+
742
+ def is_safe_filename(filename: str) -> bool:
743
+ """
744
+ Validates that the filename matches acceptable dar/par2 naming convention.
745
+ """
746
+ return archive_pattern.match(filename) is not None
747
+
748
+ def is_safe_path(path: str) -> bool:
749
+ """
750
+ Validates that the full path is absolute, has no '..'.
751
+ """
752
+ normalized = os.path.normpath(path)
753
+ filename = os.path.basename(normalized)
754
+
755
+ return (
756
+ os.path.isabs(normalized)
757
+ and '..' not in normalized.split(os.sep)
758
+ )
759
+
760
+
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: dar-backup
3
- Version: 0.7.1
3
+ Version: 0.7.2
4
4
  Summary: A script to do full, differential and incremental backups using dar. Some files are restored from the backups during verification, after which par2 redundancy files are created. The script also has a cleanup feature to remove old backups and par2 files.
5
5
  Project-URL: GPG Public Key, https://keys.openpgp.org/search?q=dar-backup@pm.me
6
6
  Project-URL: Homepage, https://github.com/per2jensen/dar-backup/tree/main/v2
@@ -693,6 +693,25 @@ Requires-Dist: argcomplete>=3.6.2
693
693
  Requires-Dist: inputimeout>=1.0.4
694
694
  Requires-Dist: jinja2>=3.1.6
695
695
  Requires-Dist: rich>=13.0.0
696
+ Provides-Extra: dev
697
+ Requires-Dist: anyio>=4.4.0; extra == 'dev'
698
+ Requires-Dist: black>=25.1.0; extra == 'dev'
699
+ Requires-Dist: coverage>=7.8.2; extra == 'dev'
700
+ Requires-Dist: h11>=0.16.0; extra == 'dev'
701
+ Requires-Dist: httpcore>=0.17.3; extra == 'dev'
702
+ Requires-Dist: psutil>=7.0.0; extra == 'dev'
703
+ Requires-Dist: pytest; extra == 'dev'
704
+ Requires-Dist: pytest-cov>=6.1.1; extra == 'dev'
705
+ Requires-Dist: pytest-timeout>=2.4.0; extra == 'dev'
706
+ Requires-Dist: pytest>=8.4.0; extra == 'dev'
707
+ Requires-Dist: requests>=2.32.2; extra == 'dev'
708
+ Requires-Dist: wheel>=0.45.1; extra == 'dev'
709
+ Requires-Dist: zipp>=3.19.1; extra == 'dev'
710
+ Provides-Extra: packaging
711
+ Requires-Dist: build>=1.2.2; extra == 'packaging'
712
+ Requires-Dist: hatch>=1.14.1; extra == 'packaging'
713
+ Requires-Dist: hatchling>=1.27.0; extra == 'packaging'
714
+ Requires-Dist: twine>=6.1.0; extra == 'packaging'
696
715
  Description-Content-Type: text/markdown
697
716
 
698
717
  <!-- markdownlint-disable MD024 -->
@@ -700,15 +719,16 @@ Description-Content-Type: text/markdown
700
719
 
701
720
  **Reliable DAR backups, automated in clean Python**
702
721
 
703
- [![codecov](https://codecov.io/gh/per2jensen/dar-backup/branch/main/graph/badge.svg)](https://codecov.io/gh/per2jensen/dar-backup)
722
+ [![Codecov](https://codecov.io/gh/per2jensen/dar-backup/branch/main/graph/badge.svg)](https://codecov.io/gh/per2jensen/dar-backup)
723
+ [![Snyk Vuln findings](https://snyk.io/test/github/per2jensen/dar-backup/badge.svg)](https://snyk.io/test/github/per2jensen/dar-backup)
704
724
  ![CI](https://github.com/per2jensen/dar-backup/actions/workflows/py-tests.yml/badge.svg)
705
725
  [![PyPI version](https://img.shields.io/pypi/v/dar-backup.svg)](https://pypi.org/project/dar-backup/)
706
- [![PyPI total Downloads](https://img.shields.io/badge/dynamic/json?color=blue&label=Total%20Downloads&query=total&url=https%3A%2F%2Fraw.githubusercontent.com%2Fper2jensen%2Fdar-backup%2Fmain%2Fdownloads.json)](https://pypi.org/project/dar-backup/)
726
+ [![PyPI downloads](https://img.shields.io/badge/dynamic/json?color=blue&label=PyPI%20downloads&query=total&url=https%3A%2F%2Fraw.githubusercontent.com%2Fper2jensen%2Fdar-backup%2Fmain%2Fdownloads.json)](https://pypi.org/project/dar-backup/)
707
727
  [![# clones](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/per2jensen/dar-backup/main/v2/doc/badges/badge_clones.json)](https://github.com/per2jensen/dar-backup/blob/main/v2/doc/weekly_clones.png)
708
728
  [![Milestone](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/per2jensen/dar-backup/main/v2/doc/badges/milestone_badge.json)](https://github.com/per2jensen/dar-backup/blob/main/v2/doc/weekly_clones.png)
709
729
 
710
730
  The wonderful 'dar' [Disk Archiver](https://github.com/Edrusb/DAR) is used for
711
- the heavy lifting, together with [parchive](https://github.com/Parchive/par2cmdline) suite in these scripts.
731
+ the heavy lifting, together with the [parchive](https://github.com/Parchive/par2cmdline) suite in these scripts.
712
732
 
713
733
  This is the `Python` based [**version 2**](https://github.com/per2jensen/dar-backup/tree/main/v2) of `dar-backup`.
714
734
 
@@ -770,6 +790,7 @@ This is the `Python` based [**version 2**](https://github.com/per2jensen/dar-bac
770
790
  - [Skipping cache directories](#skipping-cache-directories)
771
791
  - [Progress bar + current directory](#progress-bar-and-current-directory)
772
792
  - [Shell Autocompletion](#shell-autocompletion)
793
+ - [Easy development setup](#easy-development-setup)
773
794
  - [Todo](#todo)
774
795
  - [Known Limitations / Edge Cases](#known-limitations--edge-cases)
775
796
  - [Reference](#reference)
@@ -904,17 +925,17 @@ Setup the demo configurations and show a few operations
904
925
  <br>
905
926
 
906
927
  ```bash
907
- # see reference section for options tweaking the install
928
+ # See reference section for options tweaking the install
908
929
  demo --install
909
930
 
910
931
  # create catalog database
911
932
  manager --create-db
912
933
 
913
- # do a FULL backup of the installed backup definition called `demo`
934
+ # FULL backup as defined in backup definition `demo`
914
935
  dar-backup --full-backup
915
936
 
916
- # list the contents of the backup
917
- dar-backup --list-contents $(dar-backup --list |tail -n 1 | cut -d " " -f1)
937
+ # List the contents of the backup
938
+ dar-backup --list-contents demo_FULL_$(date '+%F')
918
939
  ```
919
940
 
920
941
  <details>
@@ -956,7 +977,7 @@ Config file: /home/user/.config/dar-backup/dar-backup.conf
956
977
 
957
978
 
958
979
 
959
- (venv) $ dar-backup --list-contents $(dar-backup --list |tail -n 1 | cut -d " " -f1)
980
+ (venv) $ dar-backup --list-contents demo_FULL_$(date '+%F')
960
981
  ========== Startup Settings ==========
961
982
  dar-backup.py: 0.7.1
962
983
  dar path: /home/user/.local/dar/bin/dar
@@ -984,10 +1005,10 @@ Config file: /home/user/.config/dar-backup/dar-backup.conf
984
1005
  Perform a restore and show the restored files
985
1006
 
986
1007
  ```bash
987
- # restore all files in the backup
988
- dar-backup --restore $(dar-backup --list |tail -n 1 | cut -d " " -f1) --verbose
1008
+ # Restore all files in the backup
1009
+ dar-backup --restore demo_FULL_$(date '+%F') --verbose
989
1010
 
990
- # prove the files have been restored to directory as configured
1011
+ # Prove the files have been restored to directory as configured
991
1012
  find $HOME/dar-backup/restore
992
1013
  ```
993
1014
 
@@ -996,7 +1017,7 @@ find $HOME/dar-backup/restore
996
1017
  <summary>🎯 --restore details</summary>
997
1018
 
998
1019
  ```bash
999
- (venv) $ dar-backup --verbose --restore $(dar-backup --list |tail -n 1 | cut -d " " -f1)
1020
+ (venv) $ dar-backup --restore demo_FULL_$(date '+%F') --verbose
1000
1021
  ========== Startup Settings ==========
1001
1022
  dar-backup.py: 0.7.1
1002
1023
  dar path: /home/user/.local/dar/bin/dar
@@ -1034,7 +1055,7 @@ PAR2 enabled: True
1034
1055
 
1035
1056
  > ✅ **Next steps**
1036
1057
  >
1037
- > Tinker with `demo's` options:
1058
+ > Play with `demo's` options:
1038
1059
  >
1039
1060
  > - --root-dir (perhaps $HOME)
1040
1061
  > - --dir-to-backup (perhaps Pictures)
@@ -1169,8 +1190,9 @@ Please review the [Code of Conduct](https://github.com/per2jensen/dar-backup/blo
1169
1190
 
1170
1191
  ## Requirements
1171
1192
 
1193
+ - A linux system
1172
1194
  - dar
1173
- - par2
1195
+ - parchive (par2)
1174
1196
  - python3
1175
1197
  - python3-venv
1176
1198
 
@@ -1275,27 +1297,25 @@ See section 15 and section 16 in the supplied "LICENSE" file.
1275
1297
 
1276
1298
  ### 2 - configuration
1277
1299
 
1278
- The dar-backup [demo](#demo-options) application can be used to demo how `dar-backup` works.
1279
- It creates some directories, installs a demo configuration file and puts a demo backup definition in place.
1300
+ The dar-backup [installer](#installer-options) application can be used to setup the needed directories for `dar-backup` to work.
1301
+ It creates necessary directories as prescribed in the config file and optionally creates manager databases.
1280
1302
 
1281
- `demo` is non-destructive and stops if some of the default directories exist.
1303
+ `installer` can also add configuration of shell auto completion.
1282
1304
 
1283
- Run `demo`
1305
+ Step 1:
1284
1306
 
1285
- ```bash
1286
- demo --install
1287
- ```
1307
+ Create a config file - [see details on config file](#config-file))
1288
1308
 
1289
- The output is
1309
+ Step 2:
1290
1310
 
1291
- ```text
1292
- Directories created: `/home/user/dar-backup/` and `/home/user/.config/dar-backup`
1293
- Config file deployed to /home/user/.config/dar-backup/dar-backup.conf
1294
- Default backup definition deployed to /home/user/.config/dar-backup/backup.d/default
1295
- 1. Now run `manager --create-db` to create the catalog database.
1296
- 2. Then you can run `dar-backup --full-backup` to create a backup.
1297
- 3. List backups with `dar-backup --list`
1298
- 4. List contents of a backup with `dar-backup --list-contents <backup-name>`
1311
+ Create one or more backup definitions - [see details on backup definitions](#backup-definition-example)
1312
+
1313
+ Step 3:
1314
+
1315
+ Run the installer:
1316
+
1317
+ ```bash
1318
+ installer --config <path to dar-backup.conf> --install-autocompletion
1299
1319
  ```
1300
1320
 
1301
1321
  ### 3 - generate catalog databases
@@ -1310,9 +1330,7 @@ manager --create-db
1310
1330
 
1311
1331
  ### 4 - give dar-backup a spin
1312
1332
 
1313
- The `demo` application has put a demo [backup definition](#backup-definition-example) in place in BACKUP.D_DIR (see [config file](#config-file)).
1314
-
1315
- You are now ready to do backups as configured in the backup definition.
1333
+ You are now ready to do backups as configured in your backup definition(s).
1316
1334
 
1317
1335
  Give `dar-backup`a spin:
1318
1336
 
@@ -1322,6 +1340,9 @@ dar-backup --full-backup --verbose
1322
1340
  # list backups
1323
1341
  dar-backup --list
1324
1342
 
1343
+ # list contents of a dar backup
1344
+ dar-backup --list-contents <TAB>... <choose a backup>
1345
+
1325
1346
  # see some examples on usage
1326
1347
  dar-backup --examples
1327
1348
 
@@ -1360,6 +1381,9 @@ Tilde `~` and environment variables can be used in the paths for various file lo
1360
1381
  ```text
1361
1382
  [MISC]
1362
1383
  LOGFILE_LOCATION=~/.dar-backup.log
1384
+ # optional parameters
1385
+ # LOGFILE_MAX_BYTES = 26214400 # 25 MB max file size is default, change as neeeded
1386
+ # LOGFILE_BACKUP_COUNT = 5 # 5 backup log files is default, change as needed
1363
1387
  MAX_SIZE_VERIFICATION_MB = 20
1364
1388
  MIN_SIZE_VERIFICATION_MB = 1
1365
1389
  NO_FILES_VERIFICATION = 5
@@ -1378,6 +1402,8 @@ TEST_RESTORE_DIR = /tmp/dar-backup/restore/
1378
1402
 
1379
1403
  [AGE]
1380
1404
  # age settings are in days
1405
+ # `cleanup` script removes archives and their .par redundancy files if older than configured.
1406
+ # `cleanup` does not remove FULL archives, unless specifically told to and a "y" is answered to "are you sure?".
1381
1407
  DIFF_AGE = 100
1382
1408
  INCR_AGE = 40
1383
1409
 
@@ -1533,6 +1559,12 @@ The name of the file is the name of the backup definition.
1533
1559
 
1534
1560
  You can use as many backup definitions as you need.
1535
1561
 
1562
+ > Note 👉
1563
+ >
1564
+ > Environment variables and tilde (~) not allowed here. `dar` does not expand them.
1565
+ >
1566
+ > See [TODO](#todo)
1567
+
1536
1568
  ```text
1537
1569
  # Switch to ordered selection mode, which means that the following
1538
1570
  # options will be considered top to bottom
@@ -1906,6 +1938,8 @@ These [.darrc](#darrc) settings make `dar` print the current directory being pro
1906
1938
 
1907
1939
  This is very useful in very long running jobs to get an indication that the backup is proceeding normally.
1908
1940
 
1941
+ The `dar` output is streamed to the `dar-backup-commands.log` file.
1942
+
1909
1943
  ### Separate log file for command output
1910
1944
 
1911
1945
  Dar-backup's log file is called `dar-backup.log`.
@@ -2067,8 +2101,54 @@ Then reload Zsh:
2067
2101
  source ~/.zshrc
2068
2102
  ```
2069
2103
 
2104
+ ## Easy development setup
2105
+
2106
+ It is very easy to have your own development environment.
2107
+
2108
+ ```bash
2109
+ git clone https://github.com/per2jensen/dar-backup.git
2110
+ cd dar-backup/v2
2111
+ ./build.py
2112
+ ```
2113
+
2114
+ This script:
2115
+
2116
+ - Creates a Python virtual environment called `venv`
2117
+ - pip install `hatch`
2118
+ - pip install the development environment as setup in pyproject.toml
2119
+
2120
+ --
2121
+
2122
+ ```text
2123
+ dev = [
2124
+ "pytest",
2125
+ "wheel>=0.45.1",
2126
+ "requests>=2.32.2",
2127
+ "coverage>=7.8.2",
2128
+ "pytest>=8.4.0",
2129
+ "pytest-cov>=6.1.1",
2130
+ "psutil>=7.0.0",
2131
+ "pytest-timeout>=2.4.0",
2132
+ "httpcore>=0.17.3",
2133
+ "h11>=0.16.0",
2134
+ "zipp>=3.19.1",
2135
+ "anyio>=4.4.0",
2136
+ "black>=25.1.0"]
2137
+ ```
2138
+
2139
+ ✅ Your environment is now ready to activate and test!
2140
+
2141
+ Activate and run the test suite:
2142
+
2143
+ ```bash
2144
+ source venv/bin/activate # activate the virtual env
2145
+ pytest # run the test suite
2146
+ ```
2147
+
2070
2148
  ## Todo
2071
2149
 
2150
+ - Perhaps look into pre-processing backup definitions. As `dar` does not expand env vars
2151
+ `dar-backup` could do so and feed the result to `dar`.
2072
2152
  - When run interactively, a progress bar during test and par2 generation would be nice.
2073
2153
  - Look into a way to move the .par2 files away from the `dar` slices, to maximize chance of good redundancy.
2074
2154
  - Add option to dar-backup to use the `dar` option `--fsa-scope none`
@@ -2233,12 +2313,12 @@ Sets up `dar-backup` according to provided config file.
2233
2313
  The installer creates the necessary backup catalog databases if `--create-db` is given.
2234
2314
 
2235
2315
  ```bash
2236
- --config Sets up `dar-backup`.
2237
- --create-db Create backup catalog databases. Add it to --config
2238
- --install-autocompletion Add bash or zsh auto completion - idempotent
2239
- --remove-autocompletion Remove the auto completion from bash or zsh
2316
+ --config Path to a config file. The configured directories will be created.
2317
+ --create-db Create backup catalog databases. Use this option with `--config`.
2318
+ --install-autocompletion Add bash or zsh auto completion - idempotent.
2319
+ --remove-autocompletion Remove the auto completion from bash or zsh.
2240
2320
  -v, --version Display version and licensing information.
2241
- -h, --help Displays usage info
2321
+ -h, --help Displays usage info.
2242
2322
  ```
2243
2323
 
2244
2324
  ### Demo options
@@ -2258,7 +2338,7 @@ Create directories:
2258
2338
  Sets up demo config files:
2259
2339
 
2260
2340
  - ~/.config/dar-backup/dar-backup.conf
2261
- - ~/.config/dar-backup/backup.d/default
2341
+ - ~/.config/dar-backup/backup.d/demo
2262
2342
 
2263
2343
  ```bash
2264
2344
  -i, --install Sets up `dar-backup`.
@@ -0,0 +1,25 @@
1
+ dar_backup/.darrc,sha256=-aerqivZmOsW_XBCh9IfbYTUvw0GkzDSr3Vx4GcNB1g,2113
2
+ dar_backup/Changelog.md,sha256=9zNIpgLQUpa3_QG172ZR-Bz7Uf5ngnhpW_GQw7GGJu4,11532
3
+ dar_backup/README.md,sha256=HWrcpd_nhzhL2quLlxp7Xd0i05GxZvJMu5nSw8lJKLk,59901
4
+ dar_backup/__about__.py,sha256=za-A4zLS5L2_gT8XDhJozoBlWjX9ihNSMrswofJc5dE,344
5
+ dar_backup/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
6
+ dar_backup/clean_log.py,sha256=JVrc3VArmKOxr3n5P3u_9hB4G6nsabkR3o7cRIAZ7FI,5493
7
+ dar_backup/cleanup.py,sha256=_ggDcpMCB1MhXStvYussp_PdGfhIFtEutT5BNrNkMSY,13297
8
+ dar_backup/command_runner.py,sha256=IUPYYBsaDFBp0q81Rt2xB9ucuK9eu5bXziRKXhI5YZM,4500
9
+ dar_backup/config_settings.py,sha256=2UAHvatrVO4ark6lCn2Q7qBvZN6DUMK2ftlNrKpzlWc,5792
10
+ dar_backup/dar-backup.conf,sha256=46V2zdvjj_aThFY11wWlffjmoiChYmatdf5DXCsmmao,1208
11
+ dar_backup/dar-backup.conf.j2,sha256=z3epGo6nB_Jh3liTOp93wJO_HKUsf7qghe9cdtFH7cY,2021
12
+ dar_backup/dar_backup.py,sha256=pgUQ1wZ5_yocRJJDRa4AkVyI7I8Z5WBY5vD5a4ASkmA,43001
13
+ dar_backup/dar_backup_systemd.py,sha256=PwAc2H2J3hQLWpnC6Ib95NZYtB2G2NDgkSblfLj1n10,3875
14
+ dar_backup/demo.py,sha256=bxEq_nJwHuQydERPppkvhotg1fdwBX_CE33m5fX_kxw,7945
15
+ dar_backup/demo_backup_def.j2,sha256=hQW2Glp0QGV3Kt8cwjS0mpOCdyzjVlpgbgL6LpXTKJA,1793
16
+ dar_backup/exceptions.py,sha256=dBQwgKUilgfb1Tnlm_m1Dl1IqoY75L0n1uRkm0DBVTk,158
17
+ dar_backup/installer.py,sha256=xSXh77qquIZbUTSY3AbhERQbS7bnrPE__M_yqpszdhM,6883
18
+ dar_backup/manager.py,sha256=d1zliFpSuWc8JhjKqLMC-xOhp5kSIcfeGkMZOVuZcM0,27143
19
+ dar_backup/rich_progress.py,sha256=SfwFxebBl6jnDQMUQr4McknkW1yQWaJVo1Ju1OD3okA,3221
20
+ dar_backup/util.py,sha256=iTOGsZyIdkvh9tIu7hD_IXi-9HO6GhVgqact5GGInEY,26063
21
+ dar_backup-0.7.2.dist-info/METADATA,sha256=IRUzpVBrqUEvX1-glxHhwt-B288N3EfjYOAxoau-M4E,102497
22
+ dar_backup-0.7.2.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
23
+ dar_backup-0.7.2.dist-info/entry_points.txt,sha256=pOK9M8cHeAcGIatrYzkm_1O89kPk0enyYONALYjFBx4,286
24
+ dar_backup-0.7.2.dist-info/licenses/LICENSE,sha256=OXLcl0T2SZ8Pmy2_dmlvKuetivmyPd5m1q-Gyd-zaYY,35149
25
+ dar_backup-0.7.2.dist-info/RECORD,,
@@ -1,25 +0,0 @@
1
- dar_backup/.darrc,sha256=-aerqivZmOsW_XBCh9IfbYTUvw0GkzDSr3Vx4GcNB1g,2113
2
- dar_backup/Changelog.md,sha256=sf00ukQPTTqwraUf-NaGtMlWRBjgsQ2wyoEkF1rBew8,10834
3
- dar_backup/README.md,sha256=Sc1721Ar5PVj2tkhjrmyJXt3a5iCAlLSRjyYvUi8S9s,58463
4
- dar_backup/__about__.py,sha256=EL4Ktx4eIjaIDyiDjA8mHNzo4u6QvMkXc0WXCOClYhg,344
5
- dar_backup/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
6
- dar_backup/clean_log.py,sha256=tinRODoWWKvs6Ct41j2E4K6Ktw8CeZlsQVtyYioVVfQ,5492
7
- dar_backup/cleanup.py,sha256=VJdFHch21GWWGv1b_GlVK6dtxuUkHUazTVwmt-O3Gds,12691
8
- dar_backup/command_runner.py,sha256=IUPYYBsaDFBp0q81Rt2xB9ucuK9eu5bXziRKXhI5YZM,4500
9
- dar_backup/config_settings.py,sha256=1iRcurEPJObzVaYtiVRhXJi5DFT04VuLKbF9WIHbCvw,5305
10
- dar_backup/dar-backup.conf,sha256=46V2zdvjj_aThFY11wWlffjmoiChYmatdf5DXCsmmao,1208
11
- dar_backup/dar-backup.conf.j2,sha256=QdFXcuLFbz3UOOqR5SuHCJqds5cmtVnWrxW5Eg3sFwU,1871
12
- dar_backup/dar_backup.py,sha256=6bcFRHyMoy1epxQ1AqKnNk2L72wx9WifNuMZf0oFi2I,42586
13
- dar_backup/dar_backup_systemd.py,sha256=PwAc2H2J3hQLWpnC6Ib95NZYtB2G2NDgkSblfLj1n10,3875
14
- dar_backup/demo.py,sha256=bxEq_nJwHuQydERPppkvhotg1fdwBX_CE33m5fX_kxw,7945
15
- dar_backup/demo_backup_def.j2,sha256=hQW2Glp0QGV3Kt8cwjS0mpOCdyzjVlpgbgL6LpXTKJA,1793
16
- dar_backup/exceptions.py,sha256=dBQwgKUilgfb1Tnlm_m1Dl1IqoY75L0n1uRkm0DBVTk,158
17
- dar_backup/installer.py,sha256=bLDlJnGNExHElr7xp-G5o9IC9pN4jmhaPdphfK4VoAE,6492
18
- dar_backup/manager.py,sha256=ygBgxMHGRWtbSxylWsK3AugsQamWL68BAsWPjDpUEvo,26766
19
- dar_backup/rich_progress.py,sha256=rfKcE9P7hIGmmz9E1zMqC98axZk9fq1o-lVPeMvH2MI,3173
20
- dar_backup/util.py,sha256=_7RLLKsOoGktAgajjSYL29rs4jBJzRw0F21FXXLBnDE,24154
21
- dar_backup-0.7.1.dist-info/METADATA,sha256=sNYO2Vx8AIrwhxtISFXHEBpNpviui65IIlHo8hv1fI0,100213
22
- dar_backup-0.7.1.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
23
- dar_backup-0.7.1.dist-info/entry_points.txt,sha256=pOK9M8cHeAcGIatrYzkm_1O89kPk0enyYONALYjFBx4,286
24
- dar_backup-0.7.1.dist-info/licenses/LICENSE,sha256=OXLcl0T2SZ8Pmy2_dmlvKuetivmyPd5m1q-Gyd-zaYY,35149
25
- dar_backup-0.7.1.dist-info/RECORD,,