miniflux_sanity 0.2.0

data/README.md

@@ -0,0 +1,64 @@
+# miniflux-sanity
+
+A Ruby command-line utility to mark older entries as read on your Miniflux app. Defaults to items older than 30 days. Switch to ~1 day to wake up to a fresh feed every day.
+
+![A screenshot from my Terminal showcasing the utility in action](./assets/miniflux-sanity_cli.png)
+
+__Before:__
+
+![A screenshot from my Miniflux app showing 5031 unread items](./assets/miniflux-sanity_before.png)
+
+__After:__
+
+![A screenshot from my Miniflux app showing 516 unread items](./assets/miniflux-sanity_after.png)
+
+## Motivation
+
+If I haven't read something in the preceding month, it's unlikely I ever will. Miniflux doesn't offer an _archive_ option so we mark entries as read instead. All it really does is offer me a saner overview of "unread" items at the top.
+
+As is usually the case for me, I wanted to build something meaningful as I pick up Ruby again. This was a small use-case that was a good first challenge to tackle.
+
+The code is admittedly not perfect. I welcome any constructive criticism or feedback, more so if you are a Ruby enthusiast.
+
+## Feature-set
+
+- Uses token authentication
+- Supports cloud and self-hosted Miniflux apps
+- Configure number of days before which to mark items as read
+- Resumes marking as read if interrupted
+
+## To-do
+
+- [ ] Publish as a RubyGem for easier usage (see the [feature/rubygem](https://github.com/hirusi/miniflux-sanity/tree/feature/rubygem) branch)
+- [ ] Unit testing
+- [ ] Resume fetching if command crashes in between
+- [ ] If an item is starred _and_ unread, don't mark it as read.
+    - This could lend itself to a nice workflow where my "to-read" can be starred while scanning through items.
+
+## Goals
+
+- Get comfortable with Ruby's syntax
+- Work with `Class`, `Module`, `dotenv` etc.
+- Work with `JSON`
+- Work with Ruby's `File` API
+- Interact with an API using an HTTP library
+
+## Usage
+
+You must have Ruby available on your system/shell.
+
+Install by running `gem install miniflux_sanity`. All command line options can be viewed by running `miniflux_sanity --help`.
+
+## Development
+
+The Ruby version is specified in `.ruby-version`. `rbenv` is able to read and set the correct local version in your shell.
+
+- `git clone git@github.com:hirusi/miniflux-sanity.git`
+- `cd miniflux-sanity`
+- `cp .env.template .env`
+- Update the `.env` file as required.
+    - You'll need a token from your Miniflux app under `Settings > API Keys -> Create a new API key`
+- Install the dependencies: `bundle`
+- Run the utility: `bundle exec ruby main.rb`
+
+If you have a Docker setup to contribute using Alpine OS as its base, I'd be very happy to merge your PR.

data/Rakefile

@@ -0,0 +1 @@
+require "bundler/gem_tasks"

data/bin/miniflux_sanity

@@ -0,0 +1,53 @@
+#!/usr/bin/env ruby
+
+require 'miniflux_sanity'
+require 'rationalist'
+
+argv = Rationalist.parse(ARGV, **{
+    :string => [
+        'host',
+        'token'
+    ],
+    :default => {
+        :host => 'https://reader.miniflux.app/',
+        :days => 30
+    }
+})
+
+if argv[:help]
+    puts "miniflux_sanity is a command line utility to mark items older than specified time as read in Miniflux."
+    puts ""
+    puts "You may pass in the following arguments:"
+    puts ""
+    puts "--version     show currently installed version"
+    puts ""
+    puts "--help        show this help message"
+    puts ""
+    puts "--host        your miniflux host (optional)
+              default: https://reader.miniflux.app/"
+    puts ""
+    puts "--token       your miniflux API token (required)
+              generate from Settings > API Keys > Create a new API key"
+    puts ""
+    puts "--days        number of days before which to mark items as read (optional)
+              default: 30
+              example: 7"
+    exit true
+end
+
+if argv[:version]
+    puts "miniflux_sanity v0.2.0"
+    exit true
+end
+
+if argv[:token].nil?
+    puts "You must at least specify the API token!"
+    puts ""
+    puts "--token  (required) your miniflux API token
+                    generate from Settings > API Keys > Create a new API key"
+    exit
+end
+
+miniflux_sanity = MinifluxSanity.new token: argv[:token], host: argv[:host], days: argv[:days]
+miniflux_sanity.fetch_entries
+miniflux_sanity.mark_entries_as_read

data/lib/config.rb

@@ -0,0 +1,23 @@
+class Config
+	attr_reader :cutoff_date, :cutoff_timestamp, :auth
+
+	def initialize(host:, token:, days:)
+		require 'date'
+		@cutoff_date = Date.today - days.to_i
+		@cutoff_timestamp = @cutoff_date.to_time.to_i
+		@auth = {
+			:host => host,
+			:token => token
+		}
+	end
+	
+	def load_env
+		require 'dotenv'
+		begin
+			Dotenv.load
+			rescue
+			p 'Could not load environment variables.'
+			exit(false)
+		end
+	end
+end

data/lib/hash.rb

@@ -0,0 +1,8 @@
+class ::Hash
+	# https://stackoverflow.com/a/25990044/2464435
+	def deep_merge(second)
+
+		merger = proc { |_, v1, v2| Hash === v1 && Hash === v2 ? v1.merge(v2, &merger) : Array === v1 && Array === v2 ? v1 | v2 : [:undefined, nil, :nil].include?(v2) ? v1 : v2 }
+		merge(second.to_h, &merger)
+	end
+end

data/lib/miniflux_api.rb

@@ -0,0 +1,67 @@
+require "httparty"
+require "date"
+require_relative "hash"
+
+class MinifluxApi
+	include HTTParty
+	maintain_method_across_redirects true
+
+	def initialize(host:, token:)
+		self.class.base_uri "#{host}/v1/"
+
+		@options = {
+			:headers => {
+				"X-Auth-Token": token,
+				"Accept": "application/json"
+			}
+		}
+	end
+
+	def get_entries(before:, limit: 100, offset:, status: 'unread', direction: 'asc')
+		begin
+			custom_options = @options.deep_merge({
+				:query => {
+					:status => status,
+					:direction => direction,
+					:before => before,
+					:offset => offset,
+					:limit => limit
+				}
+			})
+			response = self.class.get("/entries", custom_options)
+			response_code = response.code.to_i
+
+			if response_code >= 400
+				raise response.parsed_response
+			else
+				response.parsed_response["entries"]
+			end
+		rescue => error
+			puts "Could not get entries from your Miniflux server. More details to follow.", error
+			exit
+		end
+	end
+
+	# Pass in an array of IDs
+	def mark_entries_read(ids:)
+		new_options = @options.deep_merge({
+			:headers => {
+				"Content-Type": "application/json"
+			},
+			:body => {
+				:entry_ids => ids,
+				:status => "read"
+			}.to_json
+		})
+
+		response = self.class.put("/entries", new_options)
+
+		if response.code.to_i == 204
+			puts "Marked entries with ID #{ids.join ", "} as read."
+		else
+			puts "Could not mark entries with ID #{ids.join ", "} as read"
+			exit(false)
+		end
+
+	end
+end

data/lib/miniflux_sanity.rb

@@ -0,0 +1,101 @@
+require "date"
+require_relative "config"
+require_relative "miniflux_api"
+require_relative "utils/cache"
+
+class MinifluxSanity
+	def initialize(token:, host:, days:)
+		# Configuration object
+		# TODO Is there a way to pass this cleanly? We're passing everything we receive, with the exact same argument names as well
+		@@config = Config.new token: token, host: host, days: days
+
+		# Set up miniflux and cache clients
+		@@miniflux_client = MinifluxApi.new host: host, token: @@config.auth[:token]
+		@@cache_client = Cache.new path: "cache.json"
+	end
+
+	def last_fetched_today?
+		if @@cache_client.last_fetched.nil?
+			false
+		else
+			Date.parse(@@cache_client.last_fetched) == Date.today
+		end
+	end
+
+	def is_older_than_cutoff?(published_at:)
+		if Date.parse(published_at).to_time.to_i > @@config.cutoff_timestamp
+			false
+		else
+			true
+		end
+	end
+	
+	def fetch_entries
+		if self.last_fetched_today?
+			puts "Last run was today, skipping fetch."
+		else
+			puts "Now collecting all unread entries before the specified date."
+		end
+
+		# We get these in blocks of 250
+		# When we hit <250, we stop because that is the last call to make!
+		size = 0
+		limit = 250
+		count = limit
+		until count < limit or self.last_fetched_today? do
+
+			entries = @@miniflux_client.get_entries before: @@config.cutoff_timestamp, offset: size, limit: limit
+
+			if entries.length < 1
+				puts "No more new entries"
+				exit true
+			end
+		
+			entries.filter do |entry|
+				# Just for some extra resilience, we make sure to check the published_at date before we filter it. This would be helpful where the Miniflux API itself has a bug with its before filter, for example.
+				unless is_older_than_cutoff? published_at: entry["published_at"]
+					true
+				else
+					false
+				end
+			end
+		
+			count = entries.count
+			size = size + count
+			puts "Fetched #{size} entries."
+		
+			@@cache_client.last_fetched = Date.today.to_s
+			@@cache_client.size = size
+			@@cache_client.add_entries_to_file data: entries
+		
+			unless count < limit
+				puts "Fetching more..."
+			end
+		end
+	end
+
+	def mark_entries_as_read
+		start = 0
+		interval = 10
+		cached_data = @@cache_client.read_from_file
+
+		while @@cache_client.size != 0 do
+			stop = start + interval
+
+			# For every 10 entries, mark as read.
+			# Reduce size and remove entries accordingly in our file.
+			filtered_data = cached_data["data"][start...stop]
+			
+			ids_to_mark_read = filtered_data.map { |entry| entry["id"] }
+
+			@@miniflux_client.mark_entries_read ids: ids_to_mark_read
+
+			@@cache_client.size -= interval
+			@@cache_client.remove_entries_from_file ids: ids_to_mark_read
+
+			start += interval
+
+			puts "#{@@cache_client.size} entries left to be mark as read."
+		end
+	end
+end

data/lib/utils/cache.rb

@@ -0,0 +1,67 @@
+require "json"
+class Cache
+	attr_accessor :size, :last_fetched
+
+	def initialize(path:)
+		@config = {
+			:path => path
+		}
+
+		@size = File.readable?(@config[:path]) ? JSON.parse(File.read(@config[:path]).to_s)["data"].count : 0
+
+		@last_fetched = File.readable?(@config[:path]) ? JSON.parse(File.read(@config[:path]).to_s)["last_fetched"] : nil
+	end
+
+	def read_from_file
+		if File.readable? @config[:path]
+			JSON.parse(File.read(@config[:path]).to_s)
+		else
+			return {
+				"size" => 0,
+				"last_fetched" => nil,
+				"data" => Array.new
+			}
+		end
+	end
+
+	def remove_entries_from_file(ids:)
+		cache = self.read_from_file
+
+		ids.each do |id|
+			# TODO We could implement a search algo like binary search for performance here
+			cache["data"].each do |cached_entry|
+				if cached_entry["id"].to_i == id.to_i
+					cache["data"].delete cached_entry
+				end
+			end
+		end
+
+		self.size = cache["data"].count
+		self.write_to_file data: cache
+	end
+
+	def add_entries_to_file(data:)
+		cache = self.read_from_file
+		new_entries_count = 0
+
+		# If an entry doesn't exist in the cache, we add it.
+		data.each do |new_entry|
+			unless cache["data"].find_index { |cache_entry| cache_entry["id"] == new_entry["id"] }
+				cache["data"].push (new_entry.filter { |key| key != "content" })
+				new_entries_count += 1
+			end
+		end
+
+		self.size = cache["data"].count
+		self.last_fetched = Date.today
+		self.write_to_file data: cache
+
+		puts "#{new_entries_count} new entries were written to cache."
+	end
+
+	def write_to_file(data:)
+		data["size"] = @size
+		data["last_fetched"] = @last_fetched
+		File.write(@config[:path], data.to_json)
+	end
+end

data/miniflux_sanity.gemspec

@@ -0,0 +1,24 @@
+Gem::Specification.new do |s|
+    s.name                        = 'miniflux_sanity'
+    s.version                     = '0.2.0'
+    s.required_ruby_version       = Gem::Requirement.new(">= 2.7.1")
+    s.date                        = '2020-09-11'
+    s.summary                     = "Mark items older than specified time as read in Miniflux."
+    s.description                 = "Command line utility to mark items older than specified time as read in Miniflux."
+    s.authors                     = ["hirusi"]
+    s.email                       = 'hello@rusingh.com'
+    s.license                     = 'AGPL-3.0'
+    
+    s.homepage                    = 'https://github.com/hirusi/miniflux-sanity'
+    s.metadata["homepage_uri"]    = s.homepage
+    s.metadata["source_code_uri"] = s.homepage
+
+    # Specify which files should be added to the gem when it is released.
+    # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
+    s.files                       = Dir.chdir(File.expand_path('..', __FILE__)) do
+      `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
+    end
+    s.bindir                      = "bin"
+    s.executables                 = s.files.grep(%r{^bin/}) { |f| File.basename(f) }
+    s.require_paths               = ["lib"]
+  end

metadata

@@ -0,0 +1,65 @@
+--- !ruby/object:Gem::Specification
+name: miniflux_sanity
+version: !ruby/object:Gem::Version
+  version: 0.2.0
+platform: ruby
+authors:
+- hirusi
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2020-09-11 00:00:00.000000000 Z
+dependencies: []
+description: Command line utility to mark items older than specified time as read
+  in Miniflux.
+email: hello@rusingh.com
+executables:
+- miniflux_sanity
+extensions: []
+extra_rdoc_files: []
+files:
+- ".editorconfig"
+- ".gitignore"
+- ".ruby-version"
+- CODE_OF_CONDUCT.md
+- Gemfile
+- Gemfile.lock
+- LICENSE
+- README.md
+- Rakefile
+- assets/miniflux-sanity_after.png
+- assets/miniflux-sanity_before.png
+- assets/miniflux-sanity_cli.png
+- bin/miniflux_sanity
+- lib/config.rb
+- lib/hash.rb
+- lib/miniflux_api.rb
+- lib/miniflux_sanity.rb
+- lib/utils/cache.rb
+- miniflux_sanity.gemspec
+homepage: https://github.com/hirusi/miniflux-sanity
+licenses:
+- AGPL-3.0
+metadata:
+  homepage_uri: https://github.com/hirusi/miniflux-sanity
+  source_code_uri: https://github.com/hirusi/miniflux-sanity
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 2.7.1
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.1.2
+signing_key: 
+specification_version: 4
+summary: Mark items older than specified time as read in Miniflux.
+test_files: []