kamal-image-retention 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/README.md +111 -0
- data/exe/kamal-retain-images +5 -0
- data/lib/kamal_image_retention/cli.rb +49 -0
- data/lib/kamal_image_retention/docker_host.rb +82 -0
- data/lib/kamal_image_retention/registry_api.rb +42 -0
- data/lib/kamal_image_retention/registry_retention.rb +197 -0
- data/lib/kamal_image_retention/releases.rb +38 -0
- data/lib/kamal_image_retention.rb +18 -0
- metadata +63 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: d0a48138263317ddaa8a5ca23da6beaeaf6a340b011024ca13943ca08837f29b
|
|
4
|
+
data.tar.gz: 846384090f8dfe614ec44c7e0fd792c2ecf1c63f6c7027e152fe421e7374b9c0
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 755b66a99e784b76262ede4876b43bee45fba8d0351444c3fb79e863f98324d85d85bf107193b2947370da0ecfc2f12b131c39e27c06e03587f3de1d34c3e495
|
|
7
|
+
data.tar.gz: 6450c797086b934f67bb7f31ff8bca23d8513077df44aa6cf651c390c099f20bca947171ecf7dda8ebc46ac575574bd59068bb78a4d2f94cc82eccfa95075855
|
data/README.md
ADDED
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
# Kamal image retention
|
|
2
|
+
|
|
3
|
+
Kamal prunes old containers and images on your application servers. A copy of each
|
|
4
|
+
release can still take up disk space on the machine that built it and in your
|
|
5
|
+
Docker registry. This gem cleans up those copies for one Kamal application at a
|
|
6
|
+
time. Run it manually or from a hook after each deploy.
|
|
7
|
+
|
|
8
|
+
## Installation
|
|
9
|
+
|
|
10
|
+
Add the gem to the Gemfile you use to run Kamal. It requires Ruby 3.2 or newer
|
|
11
|
+
and declares a dependency on Kamal `~> 2.12`.
|
|
12
|
+
|
|
13
|
+
```ruby
|
|
14
|
+
group :deploy, optional: true do
|
|
15
|
+
gem "kamal-image-retention", "~> 0.1.1", require: false
|
|
16
|
+
end
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
On the machine where you run deployments, enable the group and install it:
|
|
20
|
+
|
|
21
|
+
```sh
|
|
22
|
+
bundle config set --local with deploy
|
|
23
|
+
bundle install
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
The command needs local Docker access and SSH access to Docker on the app hosts.
|
|
27
|
+
It reads the image name, registry, hosts and SSH settings from the same Kamal
|
|
28
|
+
configuration you use to deploy.
|
|
29
|
+
|
|
30
|
+
## Preview a cleanup
|
|
31
|
+
|
|
32
|
+
From the application's Kamal directory, run:
|
|
33
|
+
|
|
34
|
+
```sh
|
|
35
|
+
bundle exec kamal-retain-images -d staging
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Use your destination name, or omit `-d staging` if you deploy with just
|
|
39
|
+
`config/deploy.yml`. Pass `-c FILE` if you keep the base config elsewhere.
|
|
40
|
+
The preview counts eligible tags without removing images. Example output for
|
|
41
|
+
an application with a supported registry:
|
|
42
|
+
|
|
43
|
+
```text
|
|
44
|
+
registry.example.com/team/app registry {"obsolete_tags":3}
|
|
45
|
+
registry.example.com/team/app build machine {"obsolete_tags":2}
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
Here, three registry tags and two local tags are eligible for removal. To apply
|
|
49
|
+
the cleanup to the same destination, run:
|
|
50
|
+
|
|
51
|
+
```sh
|
|
52
|
+
bundle exec kamal-retain-images -d staging --apply
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
## What gets kept
|
|
56
|
+
|
|
57
|
+
The gem keeps the five newest distinct builds tagged with a full Git SHA, ordered
|
|
58
|
+
by image creation time. To change the count, set Kamal's existing option in
|
|
59
|
+
`config/deploy.yml` or a destination override:
|
|
60
|
+
|
|
61
|
+
```yaml
|
|
62
|
+
retain_containers: 8
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
Images with a named tag such as `stable` and images referenced by running or
|
|
66
|
+
stopped containers on the build machine or the configured app hosts are also
|
|
67
|
+
kept. For example, if `stable` points to the eighth-newest build and your limit
|
|
68
|
+
is five, that build survives alongside the newest five. Shared registry layers
|
|
69
|
+
stay available to the other repositories that use them.
|
|
70
|
+
|
|
71
|
+
## Run after each deployment
|
|
72
|
+
|
|
73
|
+
Add this block to your executable `.kamal/hooks/post-deploy` shell script.
|
|
74
|
+
If you are creating the file, start it with `#!/bin/sh` and run
|
|
75
|
+
`chmod +x .kamal/hooks/post-deploy` once.
|
|
76
|
+
|
|
77
|
+
```sh
|
|
78
|
+
case "${KAMAL_COMMAND:-}" in
|
|
79
|
+
deploy|redeploy)
|
|
80
|
+
bundle exec kamal-retain-images --apply
|
|
81
|
+
;;
|
|
82
|
+
esac
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
The hook inherits `KAMAL_DESTINATION` from Kamal. Each invocation runs once
|
|
86
|
+
and exits. [Kamal hook documentation](https://kamal-deploy.org/docs/hooks/overview/)
|
|
87
|
+
|
|
88
|
+
## Registry support
|
|
89
|
+
|
|
90
|
+
Registry cleanup supports a Kamal-managed `kamal-docker-registry` container
|
|
91
|
+
running Distribution v3 with a single Docker volume at `/var/lib/registry` and
|
|
92
|
+
its default configuration. The registry can run locally on the build machine
|
|
93
|
+
or on an app host selected for the deployment. With a hosted registry such as
|
|
94
|
+
GHCR, the gem cleans up the build machine and leaves registry retention to the
|
|
95
|
+
provider.
|
|
96
|
+
|
|
97
|
+
When registry cleanup has work to do, pushes and pulls to **all repositories
|
|
98
|
+
on that registry** pause during collection. A recent upload defers collection.
|
|
99
|
+
If collection is interrupted, rerun with `--apply` from the same build machine
|
|
100
|
+
and user account so the gem can read its recovery state.
|
|
101
|
+
|
|
102
|
+
## Development
|
|
103
|
+
|
|
104
|
+
```sh
|
|
105
|
+
bundle install
|
|
106
|
+
bundle exec ruby -Ilib test/retention_test.rb
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
## Author
|
|
110
|
+
|
|
111
|
+
Built by [Anton Kopylov](https://www.kopylov.net).
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module KamalImageRetention
|
|
4
|
+
module CLI
|
|
5
|
+
def self.run(argv)
|
|
6
|
+
destination = ENV["KAMAL_DESTINATION"].presence
|
|
7
|
+
config_file = "config/deploy.yml"
|
|
8
|
+
apply = false
|
|
9
|
+
OptionParser.new do |opts|
|
|
10
|
+
opts.banner = "Usage: bin/retain-images [-d destination] [-c config_file] [--apply]"
|
|
11
|
+
opts.on("-d DESTINATION") { |value| destination = value }
|
|
12
|
+
opts.on("-c FILE") { |value| config_file = value }
|
|
13
|
+
opts.on("--apply") { apply = true }
|
|
14
|
+
end.parse!(argv)
|
|
15
|
+
abort "Unexpected argument: #{argv.first}" unless argv.empty?
|
|
16
|
+
|
|
17
|
+
config = Kamal::Configuration.create_from(config_file: Pathname(config_file), destination: destination)
|
|
18
|
+
abort "Kamal service differs from hook" if ENV["KAMAL_SERVICE"].present? && ENV["KAMAL_SERVICE"] != config.service
|
|
19
|
+
selected = ENV.fetch("KAMAL_HOSTS", "").split(",")
|
|
20
|
+
abort "Hook hosts differ from configuration" unless (selected - config.all_hosts).empty?
|
|
21
|
+
deployed = selected.empty? ? config.app_hosts : config.app_hosts & selected
|
|
22
|
+
return if deployed.empty?
|
|
23
|
+
|
|
24
|
+
local = DockerHost.new
|
|
25
|
+
hosts = config.app_hosts.to_h { |host| [ host, DockerHost.new(host, config.ssh) ] }
|
|
26
|
+
endpoint = URI.parse("http://#{config.registry.server}") if config.registry.server.present?
|
|
27
|
+
registry_host = endpoint&.hostname unless config.registry.local?
|
|
28
|
+
state = Pathname(Dir.home).join(".cache", "docker-release-retention")
|
|
29
|
+
state.mkpath
|
|
30
|
+
state.join("coordinator.lock").open("a") do |lock|
|
|
31
|
+
lock.flock(File::LOCK_EX)
|
|
32
|
+
used = local.inventory(config.run_directory)
|
|
33
|
+
hosts.each_value { |h| h.open { used.merge(h.inventory(config.run_directory)) } }
|
|
34
|
+
if endpoint && (registry_host.nil? || deployed.include?(registry_host))
|
|
35
|
+
host = registry_host ? hosts.fetch(registry_host) : local
|
|
36
|
+
registry_state = state.join(Digest::SHA256.hexdigest(config.registry.server))
|
|
37
|
+
registry_state.mkpath
|
|
38
|
+
host.open do
|
|
39
|
+
retention = RegistryRetention.new(host, config.image, endpoint.port, registry_state)
|
|
40
|
+
summary = retention.clean(config.repository, config.retain_containers, used, apply, config.run_directory)
|
|
41
|
+
puts "#{config.repository} registry #{summary.to_json}"
|
|
42
|
+
end
|
|
43
|
+
end
|
|
44
|
+
old = Releases.clean_build(local, config.repository, config.retain_containers, used, apply)
|
|
45
|
+
puts "#{config.repository} build machine #{ { obsolete_tags: old }.to_json}"
|
|
46
|
+
end
|
|
47
|
+
end
|
|
48
|
+
end
|
|
49
|
+
end
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module KamalImageRetention
|
|
4
|
+
class DockerHost
|
|
5
|
+
def initialize(host = nil, ssh = nil)
|
|
6
|
+
@host, @ssh = host, ssh
|
|
7
|
+
end
|
|
8
|
+
|
|
9
|
+
def open
|
|
10
|
+
return yield self unless @host
|
|
11
|
+
Net::SSH.start(@host, @ssh.user, **@ssh.options.except(:user)) do |session|
|
|
12
|
+
@session = session
|
|
13
|
+
yield self
|
|
14
|
+
ensure
|
|
15
|
+
@session = nil
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
def run(*args)
|
|
20
|
+
if @session
|
|
21
|
+
output = +""
|
|
22
|
+
errors = +""
|
|
23
|
+
status = {}
|
|
24
|
+
@session.exec!(args.shelljoin, status: status) do |_, stream, data|
|
|
25
|
+
(stream == :stdout ? output : errors) << data
|
|
26
|
+
end
|
|
27
|
+
success = status[:exit_code] == 0
|
|
28
|
+
else
|
|
29
|
+
env = ENV.to_h.slice("HOME", "PATH", "DOCKER_CONFIG", "DOCKER_CONTEXT", "DOCKER_HOST", "SSH_AUTH_SOCK")
|
|
30
|
+
output, errors, status = Open3.capture3(env, *args, unsetenv_others: true)
|
|
31
|
+
success = status.success?
|
|
32
|
+
end
|
|
33
|
+
raise "#{args.first} failed on #{@host || "local"}: #{errors.strip}" unless success
|
|
34
|
+
output.chomp
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
def docker(*args) = run("docker", *args)
|
|
38
|
+
|
|
39
|
+
def inspect_rows(kind, ids, format)
|
|
40
|
+
return [] if ids.empty?
|
|
41
|
+
docker(kind, "inspect", "-f", format, *ids.uniq).lines(chomp: true)
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
def check_deploy_lock(directory)
|
|
45
|
+
run("sh", "-ec", 'case "$1" in /*) dir=$1;; *) dir="$HOME/$1";; esac; for lock in "$dir"/lock-*; do [ ! -d "$lock" ] || exit 1; done', "sh", directory)
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
def inventory(directory)
|
|
49
|
+
check_deploy_lock(directory)
|
|
50
|
+
used = Set.new
|
|
51
|
+
ids = docker("ps", "-aq", "--no-trunc").split
|
|
52
|
+
rows = inspect_rows("container", ids, "{{.Image}} {{.Config.Image}}")
|
|
53
|
+
images = rows.map do |row|
|
|
54
|
+
digest, reference = row.split
|
|
55
|
+
used.merge([ digest, reference, reference.split("@").last ])
|
|
56
|
+
digest
|
|
57
|
+
end
|
|
58
|
+
inspect_rows("image", images, "{{.Id}} {{json .RepoDigests}}").each do |row|
|
|
59
|
+
digest, references = row.split(" ", 2)
|
|
60
|
+
used << digest
|
|
61
|
+
Array(JSON.parse(references)).each { |r| used << r.split("@").last }
|
|
62
|
+
end
|
|
63
|
+
used
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
def with_port(port)
|
|
67
|
+
return yield port unless @host
|
|
68
|
+
# The SSH event loop must continue servicing HTTP while the command session runs Docker.
|
|
69
|
+
Net::SSH.start(@host, @ssh.user, **@ssh.options.except(:user)) do |tunnel|
|
|
70
|
+
local_port = tunnel.forward.local(0, "127.0.0.1", port)
|
|
71
|
+
done = false
|
|
72
|
+
thread = Thread.new { tunnel.loop(0.1) { !done } }
|
|
73
|
+
begin
|
|
74
|
+
yield local_port
|
|
75
|
+
ensure
|
|
76
|
+
done = true
|
|
77
|
+
thread.join
|
|
78
|
+
end
|
|
79
|
+
end
|
|
80
|
+
end
|
|
81
|
+
end
|
|
82
|
+
end
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module KamalImageRetention
|
|
4
|
+
class RegistryAPI
|
|
5
|
+
ACCEPT = %w[application/vnd.oci.image.manifest.v1+json application/vnd.oci.image.index.v1+json
|
|
6
|
+
application/vnd.docker.distribution.manifest.v2+json application/vnd.docker.distribution.manifest.list.v2+json].join(", ")
|
|
7
|
+
|
|
8
|
+
def initialize(port)
|
|
9
|
+
@http = Net::HTTP.new("127.0.0.1", port, nil)
|
|
10
|
+
@http.open_timeout = 5
|
|
11
|
+
@http.read_timeout = 45
|
|
12
|
+
end
|
|
13
|
+
|
|
14
|
+
def open
|
|
15
|
+
attempts = 0
|
|
16
|
+
begin
|
|
17
|
+
@http.start
|
|
18
|
+
request("")
|
|
19
|
+
rescue IOError, SystemCallError, Net::HTTPBadResponse
|
|
20
|
+
@http.finish if @http.started?
|
|
21
|
+
raise if (attempts += 1) >= 30
|
|
22
|
+
sleep 0.2
|
|
23
|
+
retry
|
|
24
|
+
end
|
|
25
|
+
yield self
|
|
26
|
+
ensure
|
|
27
|
+
@http.finish if @http.started?
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
def request(path, method = "GET")
|
|
31
|
+
response = @http.send_request(method, "/v2/#{path}", nil, "Accept" => ACCEPT)
|
|
32
|
+
response.value
|
|
33
|
+
response.body
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
def content(path, digest)
|
|
37
|
+
body = request(path)
|
|
38
|
+
raise "Unexpected digest for #{path}" unless "sha256:#{Digest::SHA256.hexdigest(body)}" == digest
|
|
39
|
+
body
|
|
40
|
+
end
|
|
41
|
+
end
|
|
42
|
+
end
|
|
@@ -0,0 +1,197 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module KamalImageRetention
|
|
4
|
+
class RegistryRetention
|
|
5
|
+
ROOT = "/var/lib/registry/docker/registry/v2/repositories"
|
|
6
|
+
DIGEST = /\Asha256:[0-9a-f]{64}\z/
|
|
7
|
+
|
|
8
|
+
def initialize(host, repository, port, state, container: "kamal-docker-registry")
|
|
9
|
+
@host, @repo, @port, @state, @container = host, repository, port, state, container
|
|
10
|
+
@helper = "#{container}-retention"
|
|
11
|
+
@marker = state.join("restore-registry")
|
|
12
|
+
@journal = state.join("pending-#{Digest::SHA256.hexdigest(repository)}.json")
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
def metadata(container)
|
|
16
|
+
script = <<~'SH'
|
|
17
|
+
cd "$1"
|
|
18
|
+
find . -path '*/_manifests/revisions/*/*/link'
|
|
19
|
+
find . -path '*/_manifests/tags/*/current/link' -exec sh -ec '
|
|
20
|
+
for path do printf "%s %s\n" "$path" "$(cat "$path")"; done' sh {} +
|
|
21
|
+
SH
|
|
22
|
+
data = {}
|
|
23
|
+
@host.docker("exec", container, "sh", "-ec", script, "sh", ROOT).each_line do |line|
|
|
24
|
+
path, digest = line.split
|
|
25
|
+
repo, link = path.delete_prefix("./").split("/_manifests/", 2)
|
|
26
|
+
item = data[repo] ||= { tags: {}, revisions: Set.new }
|
|
27
|
+
if digest
|
|
28
|
+
item[:tags][link.split("/").fetch(1)] = digest
|
|
29
|
+
else
|
|
30
|
+
_, algorithm, hex, = link.split("/")
|
|
31
|
+
item[:revisions] << "#{algorithm}:#{hex}"
|
|
32
|
+
end
|
|
33
|
+
end
|
|
34
|
+
data
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
def with_api(port, &block)
|
|
38
|
+
@host.with_port(port) { |local| RegistryAPI.new(local).open(&block) }
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
def graph(roots, children)
|
|
42
|
+
found = Set.new
|
|
43
|
+
pending = roots.to_a.dup
|
|
44
|
+
until pending.empty?
|
|
45
|
+
id = pending.pop
|
|
46
|
+
pending.concat(children.fetch(id)) if found.add?(id)
|
|
47
|
+
end
|
|
48
|
+
found
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
def plan(api, entry, count, used, reference, pending)
|
|
52
|
+
tags, revisions = entry.values_at(:tags, :revisions)
|
|
53
|
+
manifests = revisions.to_h { |id| [ id, JSON.parse(api.content("#{@repo}/manifests/#{id}", id)) ] }
|
|
54
|
+
children = manifests.transform_values do |m|
|
|
55
|
+
raise "Unsupported manifest" unless m.key?("config") || m.key?("manifests")
|
|
56
|
+
(m.fetch("manifests", []) + [ m["subject"] ].compact).map { |c| c.fetch("digest") }
|
|
57
|
+
end
|
|
58
|
+
dates = {}
|
|
59
|
+
created = lambda do |id|
|
|
60
|
+
dates[id] ||= begin
|
|
61
|
+
m = manifests.fetch(id)
|
|
62
|
+
if m["config"]
|
|
63
|
+
cfg = m["config"].fetch("digest")
|
|
64
|
+
Time.iso8601(JSON.parse(api.content("#{@repo}/blobs/#{cfg}", cfg)).fetch("created"))
|
|
65
|
+
else
|
|
66
|
+
m.fetch("manifests").reject { |c| c.dig("platform", "architecture") == "unknown" }
|
|
67
|
+
.map { |c| created.call(c.fetch("digest")) }.max || raise("Missing image creation time")
|
|
68
|
+
end
|
|
69
|
+
end
|
|
70
|
+
end
|
|
71
|
+
pins = manifests.keys.select { |id| used.include?(id) || used.include?(manifests[id].dig("config", "digest")) }.to_set
|
|
72
|
+
keep = Releases.protected_tags(tags, count, used, reference, &created)
|
|
73
|
+
tags.each_value { |id| keep << id if graph([ id ], children).intersect?(pins) }
|
|
74
|
+
candidates = graph((tags.values.to_set - keep) | (pending & revisions), children)
|
|
75
|
+
live = graph((revisions - candidates) | keep | pins, children)
|
|
76
|
+
obsolete = (candidates - live).sort_by { |id| -graph([ id ], children).size }
|
|
77
|
+
blobs = obsolete.to_set
|
|
78
|
+
obsolete.each do |id|
|
|
79
|
+
m = manifests.fetch(id)
|
|
80
|
+
(m.fetch("layers", []) + [ m["config"] ].compact).each { |blob| blobs << blob.fetch("digest") }
|
|
81
|
+
end
|
|
82
|
+
[ obsolete, blobs ]
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
def verify(api, metadata)
|
|
86
|
+
metadata.each do |repo, entry|
|
|
87
|
+
blobs = Set.new
|
|
88
|
+
entry[:revisions].each do |id|
|
|
89
|
+
m = JSON.parse(api.content("#{repo}/manifests/#{id}", id))
|
|
90
|
+
(m.fetch("manifests", []) + [ m["subject"] ].compact).each do |child|
|
|
91
|
+
raise "Missing manifest in #{repo}" unless entry[:revisions].include?(child.fetch("digest"))
|
|
92
|
+
end
|
|
93
|
+
(m.fetch("layers", []) + [ m["config"] ].compact).each { |b| blobs << b.fetch("digest") }
|
|
94
|
+
end
|
|
95
|
+
blobs.each { |id| api.request("#{repo}/blobs/#{id}", "HEAD") }
|
|
96
|
+
entry[:tags].each { |tag, id| api.content("#{repo}/manifests/#{tag}", id) }
|
|
97
|
+
end
|
|
98
|
+
end
|
|
99
|
+
|
|
100
|
+
def uploads?(container)
|
|
101
|
+
!@host.docker("exec", container, "sh", "-ec", 'find "$1" -mmin -15 -path "*/_uploads/*" | head -1', "sh", ROOT).empty?
|
|
102
|
+
end
|
|
103
|
+
|
|
104
|
+
def sweep(image, candidates)
|
|
105
|
+
args = [ "run", "--rm", "--pull=never", "--network=none", "--volumes-from" ]
|
|
106
|
+
report = @host.docker(*args, "#{@container}:ro", "-e", "REGISTRY_LOG_LEVEL=error", "--entrypoint", "/bin/registry",
|
|
107
|
+
image, "garbage-collect", "--dry-run", "/etc/distribution/config.yml")
|
|
108
|
+
raise "Unexpected Distribution GC report" unless report.match?(/\d+ blobs marked, \d+ blobs and \d+ manifests eligible for deletion/)
|
|
109
|
+
eligible = report.scan(/^blob eligible for deletion: (sha256:\h{64})$/).flatten.to_set
|
|
110
|
+
links = report.scan(/^layer link eligible for deletion: #{Regexp.escape(@repo)}\/(sha256:\h{64})$/).flatten.to_set
|
|
111
|
+
foreign = @host.docker(*args, "#{@container}:ro", "--entrypoint", "sh", image, "-ec",
|
|
112
|
+
'cd "$1" && find . -path "./$2/_layers" -prune -o -path "*/_layers/sha256/*/link" -print', "sh", ROOT, @repo)
|
|
113
|
+
shared = foreign.lines.map { |line| "sha256:#{line.split("/")[-2]}" }.to_set
|
|
114
|
+
paths = []
|
|
115
|
+
candidates.each do |id|
|
|
116
|
+
raise "Invalid digest" unless id.match?(DIGEST)
|
|
117
|
+
hex = id.delete_prefix("sha256:")
|
|
118
|
+
paths << "repositories/#{@repo}/_layers/sha256/#{hex}/link" if links.include?(id)
|
|
119
|
+
paths << "blobs/sha256/#{hex[0, 2]}/#{hex}/data" if eligible.include?(id) && !shared.include?(id)
|
|
120
|
+
end
|
|
121
|
+
paths.each_slice(100) do |batch|
|
|
122
|
+
@host.docker(*args, @container, "--entrypoint", "sh", image, "-ec",
|
|
123
|
+
'cd /var/lib/registry/docker/registry/v2 && rm -f "$@"', "sh", *batch)
|
|
124
|
+
end
|
|
125
|
+
end
|
|
126
|
+
|
|
127
|
+
def write_state(path, value)
|
|
128
|
+
tmp = path.sub_ext(".tmp")
|
|
129
|
+
tmp.write(JSON.generate(value))
|
|
130
|
+
tmp.rename(path)
|
|
131
|
+
end
|
|
132
|
+
|
|
133
|
+
def recover
|
|
134
|
+
return unless @marker.exist?
|
|
135
|
+
id, helper = JSON.parse(@marker.read)
|
|
136
|
+
raise "Registry identity changed" unless @host.docker("inspect", "-f", "{{.Id}}", @container) == id
|
|
137
|
+
existing = @host.docker("ps", "-aq", "--no-trunc", "--filter", "id=#{helper}")
|
|
138
|
+
@host.docker("rm", "-f", helper) unless existing.empty?
|
|
139
|
+
@host.docker("start", id)
|
|
140
|
+
@marker.delete
|
|
141
|
+
end
|
|
142
|
+
|
|
143
|
+
def clean(reference, count, used, apply, directory)
|
|
144
|
+
recover if apply
|
|
145
|
+
ports = JSON.parse(@host.docker("inspect", "-f", "{{json .NetworkSettings.Ports}}", @container))
|
|
146
|
+
bound = Array(ports["5000/tcp"]).any? { |p| p["HostPort"] == @port.to_s && %w[127.0.0.1 0.0.0.0].include?(p["HostIp"]) }
|
|
147
|
+
raise "Unexpected registry port binding" unless bound
|
|
148
|
+
initial = metadata(@container)
|
|
149
|
+
entry = initial.fetch(@repo, { tags: {}, revisions: Set.new })
|
|
150
|
+
pending = @journal.exist? ? JSON.parse(@journal.read) : { "revisions" => [], "blobs" => [] }
|
|
151
|
+
obsolete, candidates = with_api(@port) { |api| plan(api, entry, count, used, reference, pending.fetch("revisions").to_set) }
|
|
152
|
+
summary = { obsolete_tags: entry[:tags].count { |_, id| obsolete.include?(id) } }
|
|
153
|
+
return summary unless apply && (obsolete.any? || @journal.exist?)
|
|
154
|
+
return summary.merge(deferred: "active upload") if uploads?(@container)
|
|
155
|
+
|
|
156
|
+
id, image, mounts, env = @host.docker("inspect", "-f",
|
|
157
|
+
'{{.Id}} {{.Image}} {{json .Mounts}}{{println}}{{json .Config.Env}}', @container).match(/\A(\S+) (\S+) (.+)\n(.+)\z/).captures
|
|
158
|
+
mount, *others = JSON.parse(mounts)
|
|
159
|
+
raise "Unexpected registry storage" unless others.empty? && mount&.values_at("Type", "Destination") == [ "volume", "/var/lib/registry" ]
|
|
160
|
+
raise "Custom registry configuration" if JSON.parse(env).any? { |v| v.start_with?("REGISTRY_") }
|
|
161
|
+
raise "Expected Distribution v3" unless @host.docker("exec", id, "registry", "--version").match?(%r{distribution/v3 3\.})
|
|
162
|
+
before = @host.docker("exec", id, "du", "-sk", "/var/lib/registry").split.first.to_i
|
|
163
|
+
helper = @host.docker("create", "--name", @helper, "--volumes-from", id, "-p", "127.0.0.1::5000", "--pull=never", image)
|
|
164
|
+
begin
|
|
165
|
+
write_state(@marker, [ id, helper ])
|
|
166
|
+
rescue StandardError
|
|
167
|
+
@host.docker("rm", helper)
|
|
168
|
+
raise
|
|
169
|
+
end
|
|
170
|
+
begin
|
|
171
|
+
@host.docker("stop", id)
|
|
172
|
+
@host.docker("start", helper)
|
|
173
|
+
port = Integer(@host.docker("port", helper, "5000/tcp").rpartition(":").last)
|
|
174
|
+
@host.check_deploy_lock(directory)
|
|
175
|
+
return summary.merge(deferred: "registry changed during scan") if uploads?(helper) || metadata(helper) != initial
|
|
176
|
+
candidates.merge(pending.fetch("blobs"))
|
|
177
|
+
write_state(@journal, revisions: obsolete, blobs: candidates.to_a)
|
|
178
|
+
with_api(port) do |api|
|
|
179
|
+
obsolete.each { |digest| api.request("#{@repo}/manifests/#{digest}", "DELETE") }
|
|
180
|
+
entry[:tags].delete_if { |_, digest| obsolete.include?(digest) }
|
|
181
|
+
entry[:revisions].subtract(obsolete)
|
|
182
|
+
raise "Unexpected revision changes" unless metadata(helper) == initial
|
|
183
|
+
verify(api, initial)
|
|
184
|
+
end
|
|
185
|
+
@host.docker("stop", helper)
|
|
186
|
+
sweep(image, candidates)
|
|
187
|
+
ensure
|
|
188
|
+
recover
|
|
189
|
+
end
|
|
190
|
+
raise "Unexpected revisions after collection" unless metadata(@container) == initial
|
|
191
|
+
with_api(@port) { |api| verify(api, initial) }
|
|
192
|
+
@journal.delete
|
|
193
|
+
after = @host.docker("exec", id, "du", "-sk", "/var/lib/registry").split.first.to_i
|
|
194
|
+
summary.merge(freed_bytes: (before - after) * 1024, retained_tags: entry[:tags].size)
|
|
195
|
+
end
|
|
196
|
+
end
|
|
197
|
+
end
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module KamalImageRetention
|
|
4
|
+
module Releases
|
|
5
|
+
COMMIT = /\A(?:[a-f0-9]{40}|[a-f0-9]{64})\z/
|
|
6
|
+
|
|
7
|
+
def self.protected_tags(tags, count, used, repository)
|
|
8
|
+
commits = tags.select { |tag, _| tag.match?(COMMIT) }.values.uniq
|
|
9
|
+
keep = commits.sort_by { |id| [ yield(id), id ] }.last(count).to_set
|
|
10
|
+
tags.each do |tag, id|
|
|
11
|
+
keep << id if !tag.match?(COMMIT) || used.include?(id) || used.include?("#{repository}:#{tag}")
|
|
12
|
+
end
|
|
13
|
+
keep
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
def self.clean_build(host, repository, count, used, apply)
|
|
17
|
+
ids = host.docker("image", "ls", "-aq", "--no-trunc", "--filter", "reference=#{repository}:*").split
|
|
18
|
+
tags, dates = {}, {}
|
|
19
|
+
host.inspect_rows("image", ids, "{{.Id}} {{.Created}} {{json .RepoTags}}").each do |row|
|
|
20
|
+
id, date, aliases = row.split(" ", 3)
|
|
21
|
+
dates[id] = Time.iso8601(date)
|
|
22
|
+
Array(JSON.parse(aliases)).each do |ref|
|
|
23
|
+
repo, _, tag = ref.rpartition(":")
|
|
24
|
+
tags[tag] = id if repo == repository
|
|
25
|
+
end
|
|
26
|
+
end
|
|
27
|
+
keep = protected_tags(tags, count, used, repository) { |id| dates.fetch(id) }
|
|
28
|
+
old = tags.reject { |_, id| keep.include?(id) }
|
|
29
|
+
if apply
|
|
30
|
+
old.each do |tag, id|
|
|
31
|
+
ref = "#{repository}:#{tag}"
|
|
32
|
+
host.docker("image", "rm", ref) if host.docker("image", "inspect", "-f", "{{.Id}}", ref) == id
|
|
33
|
+
end
|
|
34
|
+
end
|
|
35
|
+
old.size
|
|
36
|
+
end
|
|
37
|
+
end
|
|
38
|
+
end
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "kamal"
|
|
4
|
+
require "digest"
|
|
5
|
+
require "fileutils"
|
|
6
|
+
require "json"
|
|
7
|
+
require "net/http"
|
|
8
|
+
require "open3"
|
|
9
|
+
require "optparse"
|
|
10
|
+
require "set"
|
|
11
|
+
require "shellwords"
|
|
12
|
+
require "time"
|
|
13
|
+
|
|
14
|
+
require_relative "kamal_image_retention/docker_host"
|
|
15
|
+
require_relative "kamal_image_retention/releases"
|
|
16
|
+
require_relative "kamal_image_retention/registry_api"
|
|
17
|
+
require_relative "kamal_image_retention/registry_retention"
|
|
18
|
+
require_relative "kamal_image_retention/cli"
|
metadata
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: kamal-image-retention
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.1.1
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- Anton Kopylov
|
|
8
|
+
bindir: exe
|
|
9
|
+
cert_chain: []
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
|
+
dependencies:
|
|
12
|
+
- !ruby/object:Gem::Dependency
|
|
13
|
+
name: kamal
|
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
|
15
|
+
requirements:
|
|
16
|
+
- - "~>"
|
|
17
|
+
- !ruby/object:Gem::Version
|
|
18
|
+
version: '2.12'
|
|
19
|
+
type: :runtime
|
|
20
|
+
prerelease: false
|
|
21
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
22
|
+
requirements:
|
|
23
|
+
- - "~>"
|
|
24
|
+
- !ruby/object:Gem::Version
|
|
25
|
+
version: '2.12'
|
|
26
|
+
executables:
|
|
27
|
+
- kamal-retain-images
|
|
28
|
+
extensions: []
|
|
29
|
+
extra_rdoc_files: []
|
|
30
|
+
files:
|
|
31
|
+
- README.md
|
|
32
|
+
- exe/kamal-retain-images
|
|
33
|
+
- lib/kamal_image_retention.rb
|
|
34
|
+
- lib/kamal_image_retention/cli.rb
|
|
35
|
+
- lib/kamal_image_retention/docker_host.rb
|
|
36
|
+
- lib/kamal_image_retention/registry_api.rb
|
|
37
|
+
- lib/kamal_image_retention/registry_retention.rb
|
|
38
|
+
- lib/kamal_image_retention/releases.rb
|
|
39
|
+
homepage: https://github.com/tonic20/kamal-image-retention
|
|
40
|
+
licenses:
|
|
41
|
+
- Nonstandard
|
|
42
|
+
metadata:
|
|
43
|
+
allowed_push_host: https://rubygems.org
|
|
44
|
+
source_code_uri: https://github.com/tonic20/kamal-image-retention
|
|
45
|
+
rdoc_options: []
|
|
46
|
+
require_paths:
|
|
47
|
+
- lib
|
|
48
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
49
|
+
requirements:
|
|
50
|
+
- - ">="
|
|
51
|
+
- !ruby/object:Gem::Version
|
|
52
|
+
version: '3.2'
|
|
53
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
54
|
+
requirements:
|
|
55
|
+
- - ">="
|
|
56
|
+
- !ruby/object:Gem::Version
|
|
57
|
+
version: '0'
|
|
58
|
+
requirements: []
|
|
59
|
+
rubygems_version: 3.6.9
|
|
60
|
+
specification_version: 4
|
|
61
|
+
summary: Retain recent Kamal builds on the build machine and in a private Distribution
|
|
62
|
+
registry
|
|
63
|
+
test_files: []
|