rfsyncer 0.0.3__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- rfsyncer-0.0.3/PKG-INFO +345 -0
- rfsyncer-0.0.3/README.md +329 -0
- rfsyncer-0.0.3/pyproject.toml +69 -0
- rfsyncer-0.0.3/src/rfsyncer/__init__.py +4 -0
- rfsyncer-0.0.3/src/rfsyncer/__main__.py +4 -0
- rfsyncer-0.0.3/src/rfsyncer/cli.py +391 -0
- rfsyncer-0.0.3/src/rfsyncer/commands/__init__.py +0 -0
- rfsyncer-0.0.3/src/rfsyncer/commands/clear.py +79 -0
- rfsyncer-0.0.3/src/rfsyncer/commands/diff.py +926 -0
- rfsyncer-0.0.3/src/rfsyncer/commands/ping.py +65 -0
- rfsyncer-0.0.3/src/rfsyncer/ssh/__init__.py +0 -0
- rfsyncer-0.0.3/src/rfsyncer/ssh/connector.py +137 -0
- rfsyncer-0.0.3/src/rfsyncer/syncer.py +285 -0
- rfsyncer-0.0.3/src/rfsyncer/util/__init__.py +0 -0
- rfsyncer-0.0.3/src/rfsyncer/util/config.py +73 -0
- rfsyncer-0.0.3/src/rfsyncer/util/consts.py +34 -0
- rfsyncer-0.0.3/src/rfsyncer/util/display.py +144 -0
- rfsyncer-0.0.3/src/rfsyncer/util/exceptions.py +2 -0
- rfsyncer-0.0.3/src/rfsyncer/util/hash.py +15 -0
- rfsyncer-0.0.3/src/rfsyncer/util/logger.py +52 -0
- rfsyncer-0.0.3/src/rfsyncer/util/parser.py +16 -0
- rfsyncer-0.0.3/src/rfsyncer/util/types.py +30 -0
rfsyncer-0.0.3/PKG-INFO
ADDED
|
@@ -0,0 +1,345 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: rfsyncer
|
|
3
|
+
Version: 0.0.3
|
|
4
|
+
Summary: A symple tool to make diffs and synchronize file trees.
|
|
5
|
+
Author: Headorteil
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Requires-Dist: jinja2>=3.1.6
|
|
8
|
+
Requires-Dist: paramiko>=3.5.1
|
|
9
|
+
Requires-Dist: python-dotenv>=1.1.1
|
|
10
|
+
Requires-Dist: pyyaml>=6.0.2
|
|
11
|
+
Requires-Dist: rich>=14.0.0
|
|
12
|
+
Requires-Dist: typer>=0.16.0
|
|
13
|
+
Requires-Python: >=3.13
|
|
14
|
+
Project-URL: Repository, https://github.com/Headorteil/rfsyncer
|
|
15
|
+
Description-Content-Type: text/markdown
|
|
16
|
+
|
|
17
|
+
# Rfsyncer
|
|
18
|
+
|
|
19
|
+
Rfsyncer is a simple tool which takes as input a directory containing a file tree.
|
|
20
|
+
|
|
21
|
+
This tree can be compared and uploaded to one ore more remote hosts.
|
|
22
|
+
|
|
23
|
+
A simple tree would be :
|
|
24
|
+
|
|
25
|
+
```
|
|
26
|
+
$ tree
|
|
27
|
+
.
|
|
28
|
+
└── root
|
|
29
|
+
├── etc
|
|
30
|
+
│ ├── hosts
|
|
31
|
+
│ ├── hosts.rfsyncer
|
|
32
|
+
│ ├── passwd
|
|
33
|
+
│ └── zsh
|
|
34
|
+
│ └── zprofile
|
|
35
|
+
├── root.rfsyncer
|
|
36
|
+
└── root
|
|
37
|
+
└── .ssh
|
|
38
|
+
└── authorized_keys
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
By using `rfsyncer diff --hosts user@1.2.3.4 --hosts MyHost --root ./root`, rfsyncer will connect to both hosts (it is able to automatically parse some fields of the ssh config file) and compare the remote files and local files. By remplacing `diff` by `install` in the command line, it will resolve diffs by uploading local files to hosts.
|
|
42
|
+
|
|
43
|
+
Rfsyncer offers the possibility to use jinja2 as a templating engine if needed.
|
|
44
|
+
|
|
45
|
+
Pre-hooks are availible to offer more templating possibilities.
|
|
46
|
+
|
|
47
|
+
> [!CAUTION]
|
|
48
|
+
> Pre-hooks are run even with the diff command. They should not modify the system.
|
|
49
|
+
>
|
|
50
|
+
> Post-hooks are run only with the install command.
|
|
51
|
+
|
|
52
|
+
# Installation
|
|
53
|
+
|
|
54
|
+
`uv tool install git+https://github.com/Headorteil/rfsyncer.git`
|
|
55
|
+
|
|
56
|
+
# Configuration
|
|
57
|
+
|
|
58
|
+
With rfsyncer, everything is templatized.
|
|
59
|
+
|
|
60
|
+
## Config file
|
|
61
|
+
|
|
62
|
+
Config file is specified with the `--config`.
|
|
63
|
+
|
|
64
|
+
- `env` is populated with env vars (`./.env` file is automatically loaded)
|
|
65
|
+
- `flag` is populated with the content of `--flag`
|
|
66
|
+
|
|
67
|
+
That gives us this kind of config file :
|
|
68
|
+
```yaml
|
|
69
|
+
general:
|
|
70
|
+
key1: value1 # Arbitrary keys to template files for all hosts
|
|
71
|
+
pre_hooks:
|
|
72
|
+
- name: test
|
|
73
|
+
path: ./hooks/test.sh
|
|
74
|
+
|
|
75
|
+
hosts:
|
|
76
|
+
host1:
|
|
77
|
+
sudo: true # Use sudo to install files as root and be able to read them when in diff mode (default : false)
|
|
78
|
+
password: {{ env.HOST1_PASSWORD }} # Needed to use sudo
|
|
79
|
+
pre_hooks:
|
|
80
|
+
- name: test-host1
|
|
81
|
+
path: ./hooks/test-host1.sh
|
|
82
|
+
host2:
|
|
83
|
+
enabled: {{ flag.host2.enabled }} # Enable or disable host
|
|
84
|
+
hostname: 1.2.3.4 # IP or domain of the remote host
|
|
85
|
+
user: user # User of the ssh host
|
|
86
|
+
port: 2222 # SSh port (default : 22)
|
|
87
|
+
identityfile: # Path to ssh key
|
|
88
|
+
host_key1: false # Arbitrary keys to template files for this host
|
|
89
|
+
host_key2: value2
|
|
90
|
+
host3: {} # Can works with nothing more than host thanks to the ssh config
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
## .rfsyncer files
|
|
94
|
+
|
|
95
|
+
To customize the default behaviour on a specific file, you can create a `<file_name>.rfsyncer` in the same dir as it.
|
|
96
|
+
|
|
97
|
+
For example, if you want to customize `root/etc/hosts`, create `root/etc/hosts.rfsyncer`
|
|
98
|
+
|
|
99
|
+
- `general` is populated with the general section of the config
|
|
100
|
+
- `general.flag` is populated with the content of `--flag`
|
|
101
|
+
- `general.env` is populated with env vars (`./.env` file is automatically loaded)
|
|
102
|
+
- `host` is populated with the corresponding host section of the config with some additions :
|
|
103
|
+
- `real_hostname` is the output of `$ hostname`
|
|
104
|
+
- `hook` is populated with the results of pre-hooks
|
|
105
|
+
|
|
106
|
+
That gives us this kind of .rfsyncer file :
|
|
107
|
+
```yaml
|
|
108
|
+
{% if host.real_hostname == "IAmHost1" %}
|
|
109
|
+
enabled: false # Determines if the file should be taken into account (default : true)
|
|
110
|
+
{% endif %}
|
|
111
|
+
{% if general.key2 %}
|
|
112
|
+
templating: j2 # Enable jinja templating for the related file (default : no templating)
|
|
113
|
+
{% endif %}
|
|
114
|
+
name: dir-{{ host.host_key2 }} # If provided, will replace the original name of the file. It can even be a path.
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
> The name field can be very powerfull, especially for directories as it will impact all their childs which will follow the new path of their parent.
|
|
118
|
+
|
|
119
|
+
## Normal files
|
|
120
|
+
|
|
121
|
+
When templating is enabled for a file, jinja2 templating is applied to it.
|
|
122
|
+
|
|
123
|
+
- `general` is populated with the general section of the config
|
|
124
|
+
- `general.flag` is populated with the content of `--flag`
|
|
125
|
+
- `general.env` is populated with env vars (`./.env` file is automatically loaded)
|
|
126
|
+
- `host` is populated with the corresponding host section of the config with some additions :
|
|
127
|
+
- `real_hostname` is the output of `$ hostname`
|
|
128
|
+
- `hook` is populated with the results of pre-hooks
|
|
129
|
+
|
|
130
|
+
For example :
|
|
131
|
+
|
|
132
|
+
```jinja2
|
|
133
|
+
Hello {{ general.env.ENV_VAR }}
|
|
134
|
+
{{ general.flag.flag1 | default("default") }}
|
|
135
|
+
{{ general.key1 }}
|
|
136
|
+
{{ host.real_hostname }}
|
|
137
|
+
{% if general.key1 %}Yipee{% endif %}
|
|
138
|
+
The hook stderr was {{ hook.test.stderr }}
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
## Hooks files
|
|
142
|
+
|
|
143
|
+
### Pre hooks
|
|
144
|
+
|
|
145
|
+
They are bash scripts which are run on the remote hosts before the diff or the install.
|
|
146
|
+
|
|
147
|
+
General hooks are run before hosts specific hooks.
|
|
148
|
+
|
|
149
|
+
The hook adds fields to `.rfsyncer` and normal files with `hook.<hook name>.stdout` and `hook.<hook name>.stderr`
|
|
150
|
+
|
|
151
|
+
The pre hooks are templatized withe the following values :
|
|
152
|
+
|
|
153
|
+
- `general` is populated with the general section of the config
|
|
154
|
+
- `general.flag` is populated with the content of `--flag`
|
|
155
|
+
- `general.env` is populated with env vars (`./.env` file is automatically loaded)
|
|
156
|
+
- `host` is populated with the corresponding host section of the config with some additions :
|
|
157
|
+
- `real_hostname` is the output of `$ hostname`
|
|
158
|
+
|
|
159
|
+
For example :
|
|
160
|
+
|
|
161
|
+
```jinja2
|
|
162
|
+
{% if host.real_hostname == "zozo" %}
|
|
163
|
+
uname -m
|
|
164
|
+
{% endif %}
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
### Post hooks
|
|
168
|
+
|
|
169
|
+
They are bash scripts which are run on the remote hosts after the diff and the install.
|
|
170
|
+
|
|
171
|
+
General hooks are run before hosts specific hooks.
|
|
172
|
+
|
|
173
|
+
The post hooks are templatized withe the following values :
|
|
174
|
+
|
|
175
|
+
- `general` is populated with the general section of the config
|
|
176
|
+
- `general.flag` is populated with the content of `--flag`
|
|
177
|
+
- `general.env` is populated with env vars (`./.env` file is automatically loaded)
|
|
178
|
+
- `host` is populated with the corresponding host section of the config with some additions :
|
|
179
|
+
- `real_hostname` is the output of `$ hostname`
|
|
180
|
+
- `hook` is populated with the results of pre-hooks
|
|
181
|
+
- `paths` is populated with the results of the diff
|
|
182
|
+
|
|
183
|
+
`paths` variable has the following form :
|
|
184
|
+
```json
|
|
185
|
+
{
|
|
186
|
+
"<local path relative to the root flag>":
|
|
187
|
+
{
|
|
188
|
+
"state": "<create|update|keep|error>",
|
|
189
|
+
"remote_path": "<absolute remote path>"
|
|
190
|
+
}
|
|
191
|
+
}
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
An example of a post hook would be :
|
|
195
|
+
|
|
196
|
+
```jinja2
|
|
197
|
+
{% if paths["etc/hosts"].state == "update" %}
|
|
198
|
+
reboot
|
|
199
|
+
{% else %}
|
|
200
|
+
echo Nothing to do on {{ host.real_hostname }}
|
|
201
|
+
{% endif %}
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
# Usage
|
|
205
|
+
|
|
206
|
+
```
|
|
207
|
+
$ rfsyncer -h
|
|
208
|
+
|
|
209
|
+
Usage: rfsyncer [OPTIONS] COMMAND [ARGS]...
|
|
210
|
+
|
|
211
|
+
Rfsyncer by Headorteil 😎
|
|
212
|
+
|
|
213
|
+
|
|
214
|
+
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────╮
|
|
215
|
+
│ --config -c PATH Config path [default: /home/headorteil/.rfsyncer/config.yml] │
|
|
216
|
+
│ --processes -p INTEGER Number of processes to pop [default: 4] │
|
|
217
|
+
│ --flag -f TEXT json to pass to templating engines │
|
|
218
|
+
│ --version -V Print the tool version │
|
|
219
|
+
│ --install-completion Install completion for the current shell. │
|
|
220
|
+
│ --show-completion Show completion for the current shell, to copy it or │
|
|
221
|
+
│ customize the installation. │
|
|
222
|
+
│ --help -h Show this message and exit. │
|
|
223
|
+
╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
|
|
224
|
+
╭─ Display ───────────────────────────────────────────────────────────────────────────────────────────╮
|
|
225
|
+
│ --verbosity-level -v INTEGER RANGE [0<=x<=3] Change the logs verbosity │
|
|
226
|
+
│ [default: 2] │
|
|
227
|
+
│ --log-to-file --no-log-to-file Enable file logging (path │
|
|
228
|
+
│ defined in config) │
|
|
229
|
+
│ [default: log-to-file] │
|
|
230
|
+
│ --display --no-display -D Display things that are │
|
|
231
|
+
│ not logs nor live like │
|
|
232
|
+
│ tables or diffs │
|
|
233
|
+
│ [default: display] │
|
|
234
|
+
│ --pager -P --no-pager Display tables in less │
|
|
235
|
+
│ [default: no-pager] │
|
|
236
|
+
│ --live -l --no-live -L Display live objects like │
|
|
237
|
+
│ progress bars │
|
|
238
|
+
│ [default: live] │
|
|
239
|
+
│ --debug -d --no-debug Use max verbosity and │
|
|
240
|
+
│ print file infos with logs │
|
|
241
|
+
│ [default: no-debug] │
|
|
242
|
+
│ --color [standard|truecolor|auto|n Color system │
|
|
243
|
+
│ one|256] [default: auto] │
|
|
244
|
+
╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
|
|
245
|
+
╭─ Commands ──────────────────────────────────────────────────────────────────────────────────────────╮
|
|
246
|
+
│ ping Test connectivity to remote hosts │
|
|
247
|
+
│ install Install local tree to remote hosts │
|
|
248
|
+
│ diff Diff local tree with remote trees │
|
|
249
|
+
│ clear Clear remote hosts of rfsyncer temporary files │
|
|
250
|
+
╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
|
|
251
|
+
```
|
|
252
|
+
|
|
253
|
+
## Diff
|
|
254
|
+
|
|
255
|
+
> The diff command don't make any change to the destination file system except it will write temporary files to `/tmp/rfsyncer` and `/tmp/rfsyncer_askpass.sh` is sudo is specified.
|
|
256
|
+
|
|
257
|
+
```
|
|
258
|
+
$ rfsyncer diff -h
|
|
259
|
+
|
|
260
|
+
Usage: rfsyncer diff [OPTIONS]
|
|
261
|
+
|
|
262
|
+
Diff local tree with remote trees
|
|
263
|
+
|
|
264
|
+
|
|
265
|
+
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────╮
|
|
266
|
+
│ --root -r PATH Root path on wich make diff [default: root] │
|
|
267
|
+
│ --hosts TEXT Hosts │
|
|
268
|
+
│ [default: (all hosts defined in the config file)] │
|
|
269
|
+
│ --sudo -s --no-sudo -S Exec commands with sudo [default: no-sudo] │
|
|
270
|
+
│ --insecure -i --no-insecure -I Insecure mode : don't check host keys │
|
|
271
|
+
│ [default: no-insecure] │
|
|
272
|
+
│ --keep -k --no-keep -K Keep remote tmp dir [default: no-keep] │
|
|
273
|
+
│ --force-upload -f --no-force-upload -F Force upload to remote, may be useful with --keep │
|
|
274
|
+
│ [default: no-force-upload] │
|
|
275
|
+
│ --help -h Show this message and exit. │
|
|
276
|
+
╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
## Install
|
|
280
|
+
|
|
281
|
+
```
|
|
282
|
+
$ rfsyncer install -h
|
|
283
|
+
|
|
284
|
+
Usage: rfsyncer install [OPTIONS]
|
|
285
|
+
|
|
286
|
+
Install local tree to remote hosts
|
|
287
|
+
|
|
288
|
+
|
|
289
|
+
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────╮
|
|
290
|
+
│ --root -r PATH Root path on wich make diff [default: root] │
|
|
291
|
+
│ --hosts TEXT Hosts [default: (all hosts defined in the config file)] │
|
|
292
|
+
│ --sudo -s --no-sudo -S Exec commands with sudo [default: no-sudo] │
|
|
293
|
+
│ --insecure -i --no-insecure -I Insecure mode : don't check host keys │
|
|
294
|
+
│ [default: no-insecure] │
|
|
295
|
+
│ --keep -k --no-keep -K Keep remote tmp dir [default: no-keep] │
|
|
296
|
+
│ --help -h Show this message and exit. │
|
|
297
|
+
╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
|
|
298
|
+
```
|
|
299
|
+
|
|
300
|
+
## Clear
|
|
301
|
+
|
|
302
|
+
```
|
|
303
|
+
$ rfsyncer clear -h
|
|
304
|
+
|
|
305
|
+
Usage: rfsyncer clear [OPTIONS]
|
|
306
|
+
|
|
307
|
+
Clear remote hosts of rfsyncer temporary files
|
|
308
|
+
|
|
309
|
+
|
|
310
|
+
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────╮
|
|
311
|
+
│ --hosts TEXT Hosts [default: (all hosts defined in the config file)] │
|
|
312
|
+
│ --insecure -i --no-insecure -I Insecure mode : don't check host keys │
|
|
313
|
+
│ [default: no-insecure] │
|
|
314
|
+
│ --help -h Show this message and exit. │
|
|
315
|
+
╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
|
|
316
|
+
```
|
|
317
|
+
|
|
318
|
+
## Ping
|
|
319
|
+
|
|
320
|
+
```
|
|
321
|
+
$ rfsyncer clear -h
|
|
322
|
+
|
|
323
|
+
Usage: rfsyncer ping [OPTIONS]
|
|
324
|
+
|
|
325
|
+
Test connectivity to remote hosts
|
|
326
|
+
|
|
327
|
+
|
|
328
|
+
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────╮
|
|
329
|
+
│ --hosts TEXT Hosts [default: (all hosts defined in the config file)] │
|
|
330
|
+
│ --sudo -s --no-sudo -S Exec commands with sudo [default: no-sudo] │
|
|
331
|
+
│ --insecure -i --no-insecure -I Insecure mode : don't check host keys │
|
|
332
|
+
│ [default: no-insecure] │
|
|
333
|
+
│ --help -h Show this message and exit. │
|
|
334
|
+
╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
|
|
335
|
+
```
|
|
336
|
+
|
|
337
|
+
# Good to know
|
|
338
|
+
|
|
339
|
+
The only file types which are handled are `normal files` and `directories`
|
|
340
|
+
|
|
341
|
+
When a file (or directory) already exist on the remote host and its content is the same, it will be skipped even if its mode, owner or group are different.
|
|
342
|
+
|
|
343
|
+
When a file is created or uploaded, its owner and group will be the ones of the ssh user or root if sudo is specified.
|
|
344
|
+
|
|
345
|
+
Diffs are skipped for binary files and files heavier than 10Mo.
|
rfsyncer-0.0.3/README.md
ADDED
|
@@ -0,0 +1,329 @@
|
|
|
1
|
+
# Rfsyncer
|
|
2
|
+
|
|
3
|
+
Rfsyncer is a simple tool which takes as input a directory containing a file tree.
|
|
4
|
+
|
|
5
|
+
This tree can be compared and uploaded to one ore more remote hosts.
|
|
6
|
+
|
|
7
|
+
A simple tree would be :
|
|
8
|
+
|
|
9
|
+
```
|
|
10
|
+
$ tree
|
|
11
|
+
.
|
|
12
|
+
└── root
|
|
13
|
+
├── etc
|
|
14
|
+
│ ├── hosts
|
|
15
|
+
│ ├── hosts.rfsyncer
|
|
16
|
+
│ ├── passwd
|
|
17
|
+
│ └── zsh
|
|
18
|
+
│ └── zprofile
|
|
19
|
+
├── root.rfsyncer
|
|
20
|
+
└── root
|
|
21
|
+
└── .ssh
|
|
22
|
+
└── authorized_keys
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
By using `rfsyncer diff --hosts user@1.2.3.4 --hosts MyHost --root ./root`, rfsyncer will connect to both hosts (it is able to automatically parse some fields of the ssh config file) and compare the remote files and local files. By remplacing `diff` by `install` in the command line, it will resolve diffs by uploading local files to hosts.
|
|
26
|
+
|
|
27
|
+
Rfsyncer offers the possibility to use jinja2 as a templating engine if needed.
|
|
28
|
+
|
|
29
|
+
Pre-hooks are availible to offer more templating possibilities.
|
|
30
|
+
|
|
31
|
+
> [!CAUTION]
|
|
32
|
+
> Pre-hooks are run even with the diff command. They should not modify the system.
|
|
33
|
+
>
|
|
34
|
+
> Post-hooks are run only with the install command.
|
|
35
|
+
|
|
36
|
+
# Installation
|
|
37
|
+
|
|
38
|
+
`uv tool install git+https://github.com/Headorteil/rfsyncer.git`
|
|
39
|
+
|
|
40
|
+
# Configuration
|
|
41
|
+
|
|
42
|
+
With rfsyncer, everything is templatized.
|
|
43
|
+
|
|
44
|
+
## Config file
|
|
45
|
+
|
|
46
|
+
Config file is specified with the `--config`.
|
|
47
|
+
|
|
48
|
+
- `env` is populated with env vars (`./.env` file is automatically loaded)
|
|
49
|
+
- `flag` is populated with the content of `--flag`
|
|
50
|
+
|
|
51
|
+
That gives us this kind of config file :
|
|
52
|
+
```yaml
|
|
53
|
+
general:
|
|
54
|
+
key1: value1 # Arbitrary keys to template files for all hosts
|
|
55
|
+
pre_hooks:
|
|
56
|
+
- name: test
|
|
57
|
+
path: ./hooks/test.sh
|
|
58
|
+
|
|
59
|
+
hosts:
|
|
60
|
+
host1:
|
|
61
|
+
sudo: true # Use sudo to install files as root and be able to read them when in diff mode (default : false)
|
|
62
|
+
password: {{ env.HOST1_PASSWORD }} # Needed to use sudo
|
|
63
|
+
pre_hooks:
|
|
64
|
+
- name: test-host1
|
|
65
|
+
path: ./hooks/test-host1.sh
|
|
66
|
+
host2:
|
|
67
|
+
enabled: {{ flag.host2.enabled }} # Enable or disable host
|
|
68
|
+
hostname: 1.2.3.4 # IP or domain of the remote host
|
|
69
|
+
user: user # User of the ssh host
|
|
70
|
+
port: 2222 # SSh port (default : 22)
|
|
71
|
+
identityfile: # Path to ssh key
|
|
72
|
+
host_key1: false # Arbitrary keys to template files for this host
|
|
73
|
+
host_key2: value2
|
|
74
|
+
host3: {} # Can works with nothing more than host thanks to the ssh config
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
## .rfsyncer files
|
|
78
|
+
|
|
79
|
+
To customize the default behaviour on a specific file, you can create a `<file_name>.rfsyncer` in the same dir as it.
|
|
80
|
+
|
|
81
|
+
For example, if you want to customize `root/etc/hosts`, create `root/etc/hosts.rfsyncer`
|
|
82
|
+
|
|
83
|
+
- `general` is populated with the general section of the config
|
|
84
|
+
- `general.flag` is populated with the content of `--flag`
|
|
85
|
+
- `general.env` is populated with env vars (`./.env` file is automatically loaded)
|
|
86
|
+
- `host` is populated with the corresponding host section of the config with some additions :
|
|
87
|
+
- `real_hostname` is the output of `$ hostname`
|
|
88
|
+
- `hook` is populated with the results of pre-hooks
|
|
89
|
+
|
|
90
|
+
That gives us this kind of .rfsyncer file :
|
|
91
|
+
```yaml
|
|
92
|
+
{% if host.real_hostname == "IAmHost1" %}
|
|
93
|
+
enabled: false # Determines if the file should be taken into account (default : true)
|
|
94
|
+
{% endif %}
|
|
95
|
+
{% if general.key2 %}
|
|
96
|
+
templating: j2 # Enable jinja templating for the related file (default : no templating)
|
|
97
|
+
{% endif %}
|
|
98
|
+
name: dir-{{ host.host_key2 }} # If provided, will replace the original name of the file. It can even be a path.
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
> The name field can be very powerfull, especially for directories as it will impact all their childs which will follow the new path of their parent.
|
|
102
|
+
|
|
103
|
+
## Normal files
|
|
104
|
+
|
|
105
|
+
When templating is enabled for a file, jinja2 templating is applied to it.
|
|
106
|
+
|
|
107
|
+
- `general` is populated with the general section of the config
|
|
108
|
+
- `general.flag` is populated with the content of `--flag`
|
|
109
|
+
- `general.env` is populated with env vars (`./.env` file is automatically loaded)
|
|
110
|
+
- `host` is populated with the corresponding host section of the config with some additions :
|
|
111
|
+
- `real_hostname` is the output of `$ hostname`
|
|
112
|
+
- `hook` is populated with the results of pre-hooks
|
|
113
|
+
|
|
114
|
+
For example :
|
|
115
|
+
|
|
116
|
+
```jinja2
|
|
117
|
+
Hello {{ general.env.ENV_VAR }}
|
|
118
|
+
{{ general.flag.flag1 | default("default") }}
|
|
119
|
+
{{ general.key1 }}
|
|
120
|
+
{{ host.real_hostname }}
|
|
121
|
+
{% if general.key1 %}Yipee{% endif %}
|
|
122
|
+
The hook stderr was {{ hook.test.stderr }}
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
## Hooks files
|
|
126
|
+
|
|
127
|
+
### Pre hooks
|
|
128
|
+
|
|
129
|
+
They are bash scripts which are run on the remote hosts before the diff or the install.
|
|
130
|
+
|
|
131
|
+
General hooks are run before hosts specific hooks.
|
|
132
|
+
|
|
133
|
+
The hook adds fields to `.rfsyncer` and normal files with `hook.<hook name>.stdout` and `hook.<hook name>.stderr`
|
|
134
|
+
|
|
135
|
+
The pre hooks are templatized withe the following values :
|
|
136
|
+
|
|
137
|
+
- `general` is populated with the general section of the config
|
|
138
|
+
- `general.flag` is populated with the content of `--flag`
|
|
139
|
+
- `general.env` is populated with env vars (`./.env` file is automatically loaded)
|
|
140
|
+
- `host` is populated with the corresponding host section of the config with some additions :
|
|
141
|
+
- `real_hostname` is the output of `$ hostname`
|
|
142
|
+
|
|
143
|
+
For example :
|
|
144
|
+
|
|
145
|
+
```jinja2
|
|
146
|
+
{% if host.real_hostname == "zozo" %}
|
|
147
|
+
uname -m
|
|
148
|
+
{% endif %}
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
### Post hooks
|
|
152
|
+
|
|
153
|
+
They are bash scripts which are run on the remote hosts after the diff and the install.
|
|
154
|
+
|
|
155
|
+
General hooks are run before hosts specific hooks.
|
|
156
|
+
|
|
157
|
+
The post hooks are templatized withe the following values :
|
|
158
|
+
|
|
159
|
+
- `general` is populated with the general section of the config
|
|
160
|
+
- `general.flag` is populated with the content of `--flag`
|
|
161
|
+
- `general.env` is populated with env vars (`./.env` file is automatically loaded)
|
|
162
|
+
- `host` is populated with the corresponding host section of the config with some additions :
|
|
163
|
+
- `real_hostname` is the output of `$ hostname`
|
|
164
|
+
- `hook` is populated with the results of pre-hooks
|
|
165
|
+
- `paths` is populated with the results of the diff
|
|
166
|
+
|
|
167
|
+
`paths` variable has the following form :
|
|
168
|
+
```json
|
|
169
|
+
{
|
|
170
|
+
"<local path relative to the root flag>":
|
|
171
|
+
{
|
|
172
|
+
"state": "<create|update|keep|error>",
|
|
173
|
+
"remote_path": "<absolute remote path>"
|
|
174
|
+
}
|
|
175
|
+
}
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
An example of a post hook would be :
|
|
179
|
+
|
|
180
|
+
```jinja2
|
|
181
|
+
{% if paths["etc/hosts"].state == "update" %}
|
|
182
|
+
reboot
|
|
183
|
+
{% else %}
|
|
184
|
+
echo Nothing to do on {{ host.real_hostname }}
|
|
185
|
+
{% endif %}
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
# Usage
|
|
189
|
+
|
|
190
|
+
```
|
|
191
|
+
$ rfsyncer -h
|
|
192
|
+
|
|
193
|
+
Usage: rfsyncer [OPTIONS] COMMAND [ARGS]...
|
|
194
|
+
|
|
195
|
+
Rfsyncer by Headorteil 😎
|
|
196
|
+
|
|
197
|
+
|
|
198
|
+
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────╮
|
|
199
|
+
│ --config -c PATH Config path [default: /home/headorteil/.rfsyncer/config.yml] │
|
|
200
|
+
│ --processes -p INTEGER Number of processes to pop [default: 4] │
|
|
201
|
+
│ --flag -f TEXT json to pass to templating engines │
|
|
202
|
+
│ --version -V Print the tool version │
|
|
203
|
+
│ --install-completion Install completion for the current shell. │
|
|
204
|
+
│ --show-completion Show completion for the current shell, to copy it or │
|
|
205
|
+
│ customize the installation. │
|
|
206
|
+
│ --help -h Show this message and exit. │
|
|
207
|
+
╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
|
|
208
|
+
╭─ Display ───────────────────────────────────────────────────────────────────────────────────────────╮
|
|
209
|
+
│ --verbosity-level -v INTEGER RANGE [0<=x<=3] Change the logs verbosity │
|
|
210
|
+
│ [default: 2] │
|
|
211
|
+
│ --log-to-file --no-log-to-file Enable file logging (path │
|
|
212
|
+
│ defined in config) │
|
|
213
|
+
│ [default: log-to-file] │
|
|
214
|
+
│ --display --no-display -D Display things that are │
|
|
215
|
+
│ not logs nor live like │
|
|
216
|
+
│ tables or diffs │
|
|
217
|
+
│ [default: display] │
|
|
218
|
+
│ --pager -P --no-pager Display tables in less │
|
|
219
|
+
│ [default: no-pager] │
|
|
220
|
+
│ --live -l --no-live -L Display live objects like │
|
|
221
|
+
│ progress bars │
|
|
222
|
+
│ [default: live] │
|
|
223
|
+
│ --debug -d --no-debug Use max verbosity and │
|
|
224
|
+
│ print file infos with logs │
|
|
225
|
+
│ [default: no-debug] │
|
|
226
|
+
│ --color [standard|truecolor|auto|n Color system │
|
|
227
|
+
│ one|256] [default: auto] │
|
|
228
|
+
╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
|
|
229
|
+
╭─ Commands ──────────────────────────────────────────────────────────────────────────────────────────╮
|
|
230
|
+
│ ping Test connectivity to remote hosts │
|
|
231
|
+
│ install Install local tree to remote hosts │
|
|
232
|
+
│ diff Diff local tree with remote trees │
|
|
233
|
+
│ clear Clear remote hosts of rfsyncer temporary files │
|
|
234
|
+
╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
|
|
235
|
+
```
|
|
236
|
+
|
|
237
|
+
## Diff
|
|
238
|
+
|
|
239
|
+
> The diff command don't make any change to the destination file system except it will write temporary files to `/tmp/rfsyncer` and `/tmp/rfsyncer_askpass.sh` is sudo is specified.
|
|
240
|
+
|
|
241
|
+
```
|
|
242
|
+
$ rfsyncer diff -h
|
|
243
|
+
|
|
244
|
+
Usage: rfsyncer diff [OPTIONS]
|
|
245
|
+
|
|
246
|
+
Diff local tree with remote trees
|
|
247
|
+
|
|
248
|
+
|
|
249
|
+
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────╮
|
|
250
|
+
│ --root -r PATH Root path on wich make diff [default: root] │
|
|
251
|
+
│ --hosts TEXT Hosts │
|
|
252
|
+
│ [default: (all hosts defined in the config file)] │
|
|
253
|
+
│ --sudo -s --no-sudo -S Exec commands with sudo [default: no-sudo] │
|
|
254
|
+
│ --insecure -i --no-insecure -I Insecure mode : don't check host keys │
|
|
255
|
+
│ [default: no-insecure] │
|
|
256
|
+
│ --keep -k --no-keep -K Keep remote tmp dir [default: no-keep] │
|
|
257
|
+
│ --force-upload -f --no-force-upload -F Force upload to remote, may be useful with --keep │
|
|
258
|
+
│ [default: no-force-upload] │
|
|
259
|
+
│ --help -h Show this message and exit. │
|
|
260
|
+
╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
|
|
261
|
+
```
|
|
262
|
+
|
|
263
|
+
## Install
|
|
264
|
+
|
|
265
|
+
```
|
|
266
|
+
$ rfsyncer install -h
|
|
267
|
+
|
|
268
|
+
Usage: rfsyncer install [OPTIONS]
|
|
269
|
+
|
|
270
|
+
Install local tree to remote hosts
|
|
271
|
+
|
|
272
|
+
|
|
273
|
+
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────╮
|
|
274
|
+
│ --root -r PATH Root path on wich make diff [default: root] │
|
|
275
|
+
│ --hosts TEXT Hosts [default: (all hosts defined in the config file)] │
|
|
276
|
+
│ --sudo -s --no-sudo -S Exec commands with sudo [default: no-sudo] │
|
|
277
|
+
│ --insecure -i --no-insecure -I Insecure mode : don't check host keys │
|
|
278
|
+
│ [default: no-insecure] │
|
|
279
|
+
│ --keep -k --no-keep -K Keep remote tmp dir [default: no-keep] │
|
|
280
|
+
│ --help -h Show this message and exit. │
|
|
281
|
+
╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
|
|
282
|
+
```
|
|
283
|
+
|
|
284
|
+
## Clear
|
|
285
|
+
|
|
286
|
+
```
|
|
287
|
+
$ rfsyncer clear -h
|
|
288
|
+
|
|
289
|
+
Usage: rfsyncer clear [OPTIONS]
|
|
290
|
+
|
|
291
|
+
Clear remote hosts of rfsyncer temporary files
|
|
292
|
+
|
|
293
|
+
|
|
294
|
+
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────╮
|
|
295
|
+
│ --hosts TEXT Hosts [default: (all hosts defined in the config file)] │
|
|
296
|
+
│ --insecure -i --no-insecure -I Insecure mode : don't check host keys │
|
|
297
|
+
│ [default: no-insecure] │
|
|
298
|
+
│ --help -h Show this message and exit. │
|
|
299
|
+
╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
|
|
300
|
+
```
|
|
301
|
+
|
|
302
|
+
## Ping
|
|
303
|
+
|
|
304
|
+
```
|
|
305
|
+
$ rfsyncer clear -h
|
|
306
|
+
|
|
307
|
+
Usage: rfsyncer ping [OPTIONS]
|
|
308
|
+
|
|
309
|
+
Test connectivity to remote hosts
|
|
310
|
+
|
|
311
|
+
|
|
312
|
+
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────╮
|
|
313
|
+
│ --hosts TEXT Hosts [default: (all hosts defined in the config file)] │
|
|
314
|
+
│ --sudo -s --no-sudo -S Exec commands with sudo [default: no-sudo] │
|
|
315
|
+
│ --insecure -i --no-insecure -I Insecure mode : don't check host keys │
|
|
316
|
+
│ [default: no-insecure] │
|
|
317
|
+
│ --help -h Show this message and exit. │
|
|
318
|
+
╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
|
|
319
|
+
```
|
|
320
|
+
|
|
321
|
+
# Good to know
|
|
322
|
+
|
|
323
|
+
The only file types which are handled are `normal files` and `directories`
|
|
324
|
+
|
|
325
|
+
When a file (or directory) already exist on the remote host and its content is the same, it will be skipped even if its mode, owner or group are different.
|
|
326
|
+
|
|
327
|
+
When a file is created or uploaded, its owner and group will be the ones of the ssh user or root if sudo is specified.
|
|
328
|
+
|
|
329
|
+
Diffs are skipped for binary files and files heavier than 10Mo.
|