ike-artifactory 0.0.1pre2

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: e54d885ec980d2c49686b6f3395adbf7ab854b433823bd0a4fdf903e9887db8b
+  data.tar.gz: e4af791e8b3751311af406bd43d29589de074b7bb2284f81195631782af8dd6e
+SHA512:
+  metadata.gz: 4fdd3f97640d59e1bd33f1c66d0b16a6fdf94c7c3f76ff7c4b8ae0e8a7190fe7608498f5023f2e89220aa4a955653154e43219949e3cca14ba566ff456297a61
+  data.tar.gz: 383af6127fec80f4974c3ba60baafcabe6ceea00a2f514ef588325ff27f85e51b9341ef334b71d86b32d9f6dd0e8909f01ae06521058663a1bb82206a105ab72

data/README.md

@@ -0,0 +1,145 @@
+# IKE Artifactory
+
+This gem provides an object-oriented interface to Artifactory API for managing objects in Artfactory,
+particularly for cleaning up old Docker images.
+
+## Classes
+
+This gem implements two classes:
+
+* `IKE::Artifactory::Client`: Interfaces with the Artifactory API
+* `IKE::Artifactory::DockerCleaner`: Uses `IKE::Artifactory::Client` to implement a single method called `cleanup!` that lets you specify a path in Artifactory that has Docker container images. The `cleanup!` method will delete all images except the following:
+  * a list of tags to be excluded (`tags_to_exclude`)
+  * any images less than a certain age (`days_old`)
+  * any the most recent N images, regardless of age (`most_recent_images`)
+
+## Utility scripts
+
+Utility scripts that use these classes can be found in the `bin` directory:
+
+* `cleaner.rb` - an interface to `IKE::Artifactory::DockerCleaner`; see [README.cleaner.md](README.cleaner.md)
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'ike-artifactory-ruby'
+```
+
+And then execute:
+
+    $ bundle install
+
+Or install it yourself as:
+
+    $ gem install ike-artifactory-ruby
+
+## Usage
+
+### IKE::Artifactory::Client
+
+To create an instance of IKE::Artifactory::Client you will need to provide next parameters:
+* **server**: Artifactory server URL. 
+* **repo_key**: Repository in Artifactory server.
+* **user**: Username to be used to access repository.
+* **password**: User's password.
+
+#### Example
+```ruby
+require 'ike-artifactory'
+
+artifactory_client = IKE::Artifactory::Client.new(
+        :server => 'https://artifactory.mydomain.com',
+        :repo_key => 'repo-key-example',
+        :user => 'Ana',
+        :password => 'supersecret'
+)
+
+object_info = artifactory_client.get_object_info 'path/to/object'
+```
+
+The output will be a hash with the proprieties of the queried object:
+```ruby
+{"repo"=>"repo-key-example",
+ "path"=>"path/to/object",
+ "created"=>"2021-05-25T15:27:21.592-07:00",
+ "createdBy"=>"some-user",
+ "lastModified"=>"2021-05-25T15:27:21.592-07:00",
+ "modifiedBy"=>"other-userr",
+ "lastUpdated"=>"2021-05-25T15:27:21.592-07:00",
+ "children"=>
+  [{"uri"=>"/manifest.json", "folder"=>false},
+   {"uri"=>
+     "/sha256__4f07dd360c1b7e40c438e6437b2044bc31b4f6e5cf36b09a06b0c67e23dfc69d",
+    "folder"=>false},
+   {"uri"=>
+     "/sha256__70fb9965a23f2226fef622992fdf507b8333c61d68259766d4721cc4ba1e5dae",
+    "folder"=>false},
+   {"uri"=>
+     "/sha256__e0f9e11d6f9b3f5af2915fd4839ea0cd268ddccce28a788f54687b6a494770bb",
+    "folder"=>false}],
+ "uri"=>
+  "https://artifactory.mydomain.com:443/artifactory/api/storage/repo-key-example/path/to/object"}
+```
+
+#### Methods
+
+##### `delete_object(path)`
+Returns `true` if the object pointed by path was successfully deleted, otherwise `false`
+
+##### `get_subdirectories(path)`
+Returns a list of subdirectories of the specified `path`.
+
+##### `get_object_age(path)`
+Returns the age of the object specified by `path`, or -1 if the age of the object could not be determined (for example, if it does not exist).
+
+##### `get_object_info(path)`
+Returns a hash with the proprieties of the queried object.
+
+##### `get_subdirectory_ages(path)`
+Returns a hash whose keys are the names of the subdirectories of `path`, and whose values are the `lastModified` age in days of the directory in question.
+
+##### `get_images(path)`
+Returns a hash whose keys are the names (tags) of the Docker images found in `path`, and whose values are the age of the Docker image in question. An entry in `path` is considered to be a Docker image if it contains the file identified by the `IKE::Artifactory::Client::IMAGE_MANIFEST` constant, which is `manifest.json`.
+
+### IKE::Artifactory::DockerCleaner
+
+The constructor arguments of IKE::Artifactory::DockerCleaner are the following:
+* `repo_host`: The URL of the Artifactory host, without the repo key included
+* `repo_key` The repository to be cleaned
+* `folder`: The repository path to be cleaned. `cleanup!` only cleans a single path (directory) and does not recurse
+* `days_old`: The cutoff age for deletion of images. Any images less that `days_old` old will not be cleaned up.
+* `most_recent_images`: The number of most recent container images to keep, regardless of age
+* `tags_to_exclude`: List of Docker container tags to be excluded from deletion, regardless of age
+* `user`: The username to be used to access repository
+* `password`: User's password.
+* `log_level` (optional): Logging verbosity, from the `Logger` Ruby core class. Defaults to ::Logger::INFO.
+* `actually_delete` (optional): Whether to actually delete the images meeting the deletion criteria (truthy) or simply provide output about what would happen (falsy). Defaults to `false`.
+
+Returns an array of image tags that would have been deleted (`actually_delete` = `false`) or were deleted (`actually_delete` = `true`):
+
+```ruby
+['tag-x', 'tag-y', 'tag-z']
+```
+
+#### Example
+```ruby
+require 'ike-artifactory'
+
+images_to_delete = IKE::Artifactory::DockerCleaner.new(
+        :server => 'https://artifactory.mydomain.com',
+        :repo_key => 'repo-key-example',
+        :folder => 'path/to/folder',
+        :days_old => 30,
+        :most_recent_images => 5,
+        :tags_to_exclude => ['tag1', 'tag2', 'tag3'],
+        :user => 'Ana',
+        :password => 'supersecret'
+).cleanup!
+
+puts "Not actually deleting images, but if I did I would have deleted these:"
+images_to_delete.each do |i|
+  puts "  #{i}"
+end
+```

data/Rakefile

@@ -0,0 +1,18 @@
+begin
+  require 'bundler/setup'
+rescue LoadError
+  puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
+end
+
+require 'bundler/gem_tasks'
+require 'rake/testtask'
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << 'lib'
+  t.libs << 'test'
+  t.pattern = 'test/**/*_test.rb'
+  t.verbose = false
+end
+
+
+task default: :test

data/bin/cleaner.rb

@@ -0,0 +1,65 @@
+#!env ruby
+
+$:.unshift(File.expand_path('../../lib', __FILE__))
+
+require 'ike_artifactory'
+
+actually_delete = false
+if ARGV[0] == '--actually-delete'
+  ARGV.shift
+  actually_delete = true
+end
+
+unless [7,8].include?(ARGV.count)
+  STDERR.puts "Usage: $0 [--actually-delete] artifactory_url repo_key username password application_list image_exclude_list days_to_keep [most_recent_images_to_keep]"
+  exit 1
+end
+
+artifactory_url = ARGV.shift
+repo_key = ARGV.shift
+user = ARGV.shift
+password = ARGV.shift
+application_list = ARGV.shift
+image_exclude_list = ARGV.shift
+days_to_keep = ARGV.shift.to_i
+most_recent_images_to_keep = ARGV.shift.to_i || 10
+
+if days_to_keep <= 7
+  STDERR.puts "Invalid number of days_to_keep: #{days_to_keep}"
+  exit 2
+end
+
+apps = File.readlines(application_list).map { |line| line.chomp }
+
+unless apps.count
+  STDERR.puts "No applications listed in #{application_list}, quitting"
+  exit 2
+end
+
+images_to_keep = File.readlines(image_exclude_list).each_with_object({}) do |line, keep|
+  parts = line.chomp.split(/:/)
+  if parts.length == 2
+    keep[parts[0]] ||= []
+    keep[parts[0]] << parts[1]
+  else
+    STDERR.puts "Can't parse #{line} as image to keep, aborting"
+    exit 3
+  end
+  keep
+end
+
+apps.each do |app|
+  puts "Cleaning #{app}"
+  cleaner = IKE::Artifactory::DockerCleaner.new(
+    actually_delete: actually_delete,
+    repo_host: artifactory_url,
+    repo_key: repo_key,
+    folder: app,
+    days_old: days_to_keep,
+    tags_to_exclude: images_to_keep[app],
+    user: user,
+    password: password,
+    most_recent_images: most_recent_images_to_keep
+  )
+  cleaner.cleanup!
+end

data/lib/ike_artifactory/client.rb

@@ -0,0 +1,116 @@
+require 'time'
+require 'json'
+require 'rest-client'
+require 'uri'
+require 'pry-byebug'
+
+module IKE
+  module Artifactory
+    class Client
+
+      IMAGE_MANIFEST = 'manifest.json'
+
+      attr_accessor :server
+      attr_accessor :repo_key
+      attr_accessor :folder_path
+      attr_accessor :user
+      attr_accessor :password
+
+      def initialize(**args)
+        @server = args[:server]
+        @repo_key = args[:repo_key]
+        @user = args[:user]
+        @password = args[:password]
+
+        raise IKEArtifactoryClientNotReady.new(msg = 'Required attributes are missing. IKEArtifactoryGem not ready.') unless self.ready?
+      end
+
+      def delete_object(path)
+        fetch(path, method: :delete) do |response, request, result|
+          response.code == 204
+        end
+      end
+
+      def get_subdirectories(path)
+        get(path) do |response|
+          (response['children'] || []).select do |c|
+            c['folder']
+          end.map do |f|
+            f['uri'][1..]
+          end
+        end
+      end
+
+      def get_object_age(path)
+        get(path) do |response|
+          ( ( Time.now - Time.iso8601(response['lastModified']) ) / (24*60*60) ).to_int
+        end
+      end
+
+      def get_object_info(path)
+        get(path)
+      end
+
+      def get_subdirectory_ages(path)
+        get(path, prefix: "#{server}:443/ui/api/v1/ui/nativeBrowser/#{repo_key}") do |response|
+          (response['children'] || []).each_with_object({}) do |child, memo|
+            days_old = ( ( Time.now.to_i - (child['lastModified']/1000) ) / (24*60*60) ).to_int
+            memo[child['name']] = days_old
+            memo
+          end
+        end
+      end
+
+      def get_images(path)
+        get_subdirectory_ages(path).select do |(folder, _age)|
+          get_object_info([path, folder, IMAGE_MANIFEST].join('/'))
+        end
+      end
+
+      private
+
+      def ready?
+        if ([server, repo_key].include? nil ) || ([user, password].include? nil )
+          return false
+        end
+        true
+      end
+
+      def fetch(path, method: :get, prefix: nil)
+        retval = nil # Work around Object#stub stomping on return values
+
+        prefix ||= "#{server}/artifactory/api/storage/#{repo_key}"
+
+        RestClient::Request.execute(
+          :method => method,
+          :url => "#{prefix}/#{path}",
+          :user => user,
+          :password => password
+        ) do |response, request, result|
+          retval =
+            if block_given?
+              yield response, request, result
+            else
+              [response, request, result]
+            end
+        end
+
+        retval
+      end
+
+      def get(path, prefix: nil)
+        fetch(path, prefix: prefix) do |response, request, result|
+          if response.code == 200
+            obj = JSON.parse(response.to_str)
+            if block_given?
+              yield obj
+            else
+              obj
+            end
+          end
+        end
+      end
+
+    end
+  end
+end

data/lib/ike_artifactory/docker_cleaner.rb

@@ -0,0 +1,82 @@
+require 'uri'
+require 'logger'
+
+module IKE
+  module Artifactory
+
+    class DockerCleaner
+      attr_accessor :repo_host
+      attr_accessor :repo_key
+      attr_accessor :folder
+      attr_accessor :days_old
+      attr_accessor :tags_to_exclude
+
+      attr_reader :client  # is not tested
+      attr_reader :most_recent_images  # is not tested
+      attr_reader :logger  # is not tested. Used for testing
+      attr_reader :actually_delete
+
+      def initialize(repo_host:, repo_key:, folder:, days_old:,
+                     tags_to_exclude:, user:, password:, most_recent_images:,
+                     log_level: ::Logger::INFO, actually_delete: false)
+
+        @repo_host = repo_host
+        @repo_key = repo_key
+        @folder = folder
+        @days_old = days_old
+        @tags_to_exclude = tags_to_exclude || []
+        @most_recent_images = most_recent_images
+
+        @actually_delete = actually_delete
+
+        @client = IKE::Artifactory::Client.new(
+          :server => repo_host,
+          :repo_key => repo_key,
+          :user => user,
+          :password => password
+        )
+
+        @logger = Logger.new(STDOUT)
+        logger.level = log_level
+      end
+
+      def cleanup!
+        deleted_images = []
+        tags = client.get_images(folder).sort_by { |_k,v| v }.to_h
+
+        too_new_to_delete = tags.keys[0...most_recent_images]
+
+        logger_prefix = "#{repo_host}/#{repo_key}"
+
+        tags.each do | tag, tag_days_old |
+
+          logger.debug "#{logger_prefix}: examining #{tag}"
+
+          if tags_to_exclude.include?(tag)
+            logger.info "#{logger_prefix}: tag #{tag} is explicitly excluded from cleanup"
+            next
+          end
+
+          if too_new_to_delete.include?(tag)
+            logger.info "#{logger_prefix}: tag #{tag} is one of the #{most_recent_images} most recent tags, preserving"
+            next
+          end
+
+          if tag_days_old < days_old
+            logger.info "#{logger_prefix}: tag #{tag} is less than #{days_old} days old, preserving"
+            next
+          end
+
+          logger.info "#{logger_prefix}: removing tag #{tag}"
+          if actually_delete
+            client.delete_object "#{folder}/#{tag}"
+          else
+            logger.info("#{logger_prefix}: Not actually deleting #{tag} because actually_delete is falsy")
+          end
+          deleted_images.append(tag)
+        end
+        deleted_images
+      end
+    end
+  end
+end

data/lib/ike_artifactory/exceptions.rb

@@ -0,0 +1,9 @@
+module IKE
+  module Artifactory
+    class IKEArtifactoryClientNotReady < StandardError
+      def initialize(msg="Unknown.")
+        super
+      end
+    end
+  end
+end

data/lib/ike_artifactory/version.rb

@@ -0,0 +1,5 @@
+module IKE
+  module Artifactory
+    VERSION = "0.0.1pre2"
+  end
+end

data/lib/ike_artifactory.rb

@@ -0,0 +1,6 @@
+require 'ike_artifactory/exceptions'
+require 'ike_artifactory/version'
+require 'ike_artifactory/client'
+require 'ike_artifactory/docker_cleaner'
+
+

metadata

@@ -0,0 +1,113 @@
+--- !ruby/object:Gem::Specification
+name: ike-artifactory
+version: !ruby/object:Gem::Version
+  version: 0.0.1pre2
+platform: ruby
+authors:
+- Nick Marden
+- Jack Newton
+- Vicente Ramos Garcia
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2021-11-10 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: rest-client
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: minitest
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: pry-byebug
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: Ruby gem for managing objects in Artfactory, particularly for cleaning
+  up old Docker images
+email:
+- nmarden@avvo.com
+- jnewton@avvo.com
+- vramosgarcia@avvo.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- README.md
+- Rakefile
+- bin/cleaner.rb
+- lib/ike_artifactory.rb
+- lib/ike_artifactory/client.rb
+- lib/ike_artifactory/docker_cleaner.rb
+- lib/ike_artifactory/exceptions.rb
+- lib/ike_artifactory/version.rb
+homepage: https://github.com/internetbrands/ike-artifactory-ruby
+licenses:
+- MIT
+metadata:
+  allowed_push_host: https://rubygems.org
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">"
+    - !ruby/object:Gem::Version
+      version: 1.3.1
+requirements: []
+rubygems_version: 3.1.6
+signing_key: 
+specification_version: 4
+summary: Provides an object-oriented interface to Artifactory API.
+test_files: []