kynda 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: bbcd196d96d5162afabc30545f53d1ad3b8900bd5457428408dda2b7fcf62acf
+  data.tar.gz: 06b3a5f8679bc51dd6271eaf22aa1c606d96ffa94757cbb7d51c552b4a2eacb4
+SHA512:
+  metadata.gz: ef4d3ab95c0a428d5a0b5c565a9a0fcccdfb1f3629bbf7ca06ccf919ebacde6f0bc47623384e7eeb44b95c7fb1f99c70cb684fe74e192dc78fef558e16c3dd9e
+  data.tar.gz: 2355d1fb0fcaecbaf0142c20556212d87559aa7c5a2f2c6c45a2ab0d7836327aa1d796a60803f8318e92a1b263de74d36bcfddd299d89b0870eeb0b566c6998b

data/.gitignore

@@ -0,0 +1,5 @@
+My Clippings
+My Clippings_Amsterdam
+My Clippings_Cat.txt
+generated_files/
+kynda_old_07062020.rb

data/.travis.yml

@@ -0,0 +1,6 @@
+---
+language: ruby
+cache: bundler
+rvm:
+  - 2.7.1
+before_install: gem install bundler -v 2.1.4

data/Gemfile

@@ -0,0 +1,11 @@
+source "https://rubygems.org"
+
+# Specify your gem's dependencies in kynda.gemspec
+
+gem 'sinatra', '2.0.5'
+gem 'rerun', '0.13.0'
+
+group :test do
+	gem "rake", "~> 12.0"
+	gem "minitest", "~> 5.0"
+end

data/Gemfile.lock

@@ -0,0 +1,38 @@
+GEM
+  remote: https://rubygems.org/
+  specs:
+    ffi (1.13.1)
+    listen (3.2.1)
+      rb-fsevent (~> 0.10, >= 0.10.3)
+      rb-inotify (~> 0.9, >= 0.9.10)
+    minitest (5.14.1)
+    mustermann (1.1.1)
+      ruby2_keywords (~> 0.0.1)
+    rack (2.2.3)
+    rack-protection (2.0.5)
+      rack
+    rake (12.3.3)
+    rb-fsevent (0.10.4)
+    rb-inotify (0.10.1)
+      ffi (~> 1.0)
+    rerun (0.13.0)
+      listen (~> 3.0)
+    ruby2_keywords (0.0.2)
+    sinatra (2.0.5)
+      mustermann (~> 1.0)
+      rack (~> 2.0)
+      rack-protection (= 2.0.5)
+      tilt (~> 2.0)
+    tilt (2.0.10)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  minitest (~> 5.0)
+  rake (~> 12.0)
+  rerun (= 0.13.0)
+  sinatra (= 2.0.5)
+
+BUNDLED WITH
+   2.1.4

data/README.md

@@ -0,0 +1,36 @@
+# Kynda
+
+Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/kynda`. To experiment with that code, run `bin/console` for an interactive prompt.
+
+TODO: Delete this and the text above, and describe your gem
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'kynda'
+```
+
+And then execute:
+
+    $ bundle install
+
+Or install it yourself as:
+
+    $ gem install kynda
+
+## Usage
+
+TODO: Write usage instructions here
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/kynda.
+

data/Rakefile

@@ -0,0 +1,10 @@
+require "bundler/gem_tasks"
+require "rake/testtask"
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "test"
+  t.libs << "lib"
+  t.test_files = FileList["test/**/*_test.rb"]
+end
+
+task :default => :test

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "kynda"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start(__FILE__)

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/config.ru

@@ -0,0 +1,2 @@
+require './kynda_app'
+run Sinatra::Application

data/highlights_test.txt

@@ -0,0 +1,4 @@
+gedogen, which means “technically illegal but officially tolerated.”
+Amsterdam is famous for one thing (besides canals, and cannabis cafés, and prostitutes): the tattered, ancient, much-misunderstood word liberalism. Amsterdam is, by most accounts, the most liberal place on earth.
+Multiculturalism proved to be a failure. It was leading not to a mixed society but to a multiplicity of ghettoized communities living next to but cut off from one another: the very opposite of a “society.”
+“We were against authority and ownership. Buying real estate was for patsers.” (The word doesn’t translate well, but “establishment assholes” gets close.)

data/kynda.gemspec

@@ -0,0 +1,28 @@
+require_relative 'lib/kynda/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "kynda"
+  spec.version       = Kynda::VERSION
+  spec.authors       = ["SamGarner"]
+  spec.email         = ["samuel.w.garner@gmail.com"]
+
+  spec.summary       = %q{Kindle 'My Clippings' organization and clean up}
+  #spec.description   = %q{TODO: Write a longer description or delete this line.}
+  spec.homepage      = "https://github.com/SamGarner/Kynda/settings"
+  spec.required_ruby_version = Gem::Requirement.new(">= 2.3.0")
+
+  spec.metadata["allowed_push_host"] = "https://rubygems.org"
+
+  spec.metadata["homepage_uri"] = spec.homepage
+  spec.metadata["source_code_uri"] = "https://github.com/SamGarner/Kynda/"
+  spec.metadata["changelog_uri"] = "https://github.com/SamGarner/Kynda/"
+
+  # Specify which files should be added to the gem when it is released.
+  # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
+  spec.files         = Dir.chdir(File.expand_path('..', __FILE__)) do
+    `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
+  end
+  spec.bindir        = "exe"
+  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
+  spec.require_paths = ["lib"]
+end

data/kynda_app.rb

@@ -0,0 +1,13 @@
+require 'sinatra'
+
+get '/' do
+	erb :index
+end
+
+get '/about' do
+	erb :about
+end
+
+get '/clippings_upload' do
+	erb :clippings_upload
+end

data/kynda_commentless

@@ -0,0 +1,26 @@
+book_title_regexp = /={10}\r\n(.*)\r\n/
+kindle_clippings = "My Clippings"
+titles = File.read(kindle_clippings).scan(book_title_regexp).flatten.uniq!
+today = Time.now.strftime("%Y%m%d")
+all_highlights_filename = "All_Kindle_Highlights #{today}.txt"
+
+File.write(all_highlights_filename,"")
+
+titles.each { |title|	
+
+	if title[0] =~ /[\w]/
+		corrected_title = title
+	else
+		corrected_title = title[1..-1]
+	end
+
+	clippings = File.read(kindle_clippings).scan(
+		/#{Regexp.quote(corrected_title)}(.*)\r\n(.*)\r\n\r\n(.*)\r\n=======/).flatten
+	highlights = clippings.reject { |clipping| clipping.match(
+		/#{Regexp.quote("- Your Highlight")}/) || clipping.length < 2 }
+	File.write("#{corrected_title.strip} #{today}.txt", highlights.join("\n\n"))
+	File.write(all_highlights_filename,
+		"="*80  +"\n#{corrected_title.strip}\n\n", mode:"a")
+	File.write(all_highlights_filename,
+		highlights.join("\n\n") + "\n", mode:"a")
+	}

data/kynda_old_07072020.rb

@@ -0,0 +1,28 @@
+# **** Insert overall script steps here ****
+
+# Get list of unique book titles from Kindle clippings .txt file
+book_title_regexp = /={10}\n(.*)\n/
+titles = File.read("shortclippings.txt").scan(book_title_regexp).flatten.uniq!
+
+# Reset aka wipe the all encompassing highlights file with today's date
+today = Time.now.strftime("%Y%m%d")
+File.write("All_Kindle_Highlights #{today}.txt","")
+
+# Iterate over each title
+# create a file with highlights for each book
+# create an all-encompassing highlight file, grouped by book
+titles.each { |title| 
+	clippings = File.read("shortclippings.txt").scan(
+		/#{Regexp.quote(title)}(.*)\n(.*)\n\n(.*)\n=======/).flatten
+	highlights = clippings.reject { |clipping| clipping.match(
+		/#{Regexp.quote("- Your Highlight")}/) || clipping.length < 2 }
+	# write file for each book with associated quotes
+	File.write("#{title.strip} #{today}.txt", highlights.join("\n\n"))
+	# write file containing all kindle quotes, grouped by book
+	# writes in a break/line of ='s between books and then the book's title
+	File.write("All_Kindle_Highlights #{today}.txt",
+		"="*80  +"\n#{title.strip}\n\n", mode:"a")
+	# writes in all the highlights for the given book
+	File.write("All_Kindle_Highlights #{today}.txt",
+		highlights.join("\n\n") + "\n", mode:"a")
+	}

data/kynda_old_07082020.rb

@@ -0,0 +1,34 @@
+# **** Insert overall script steps here ****
+
+# Get list of unique book titles from Kindle clippings .txt file
+book_title_regexp = /={10}\r\n(.*)\r\n/
+kindle_clippings = "My Clippings"
+titles = File.read(kindle_clippings).scan(book_title_regexp).flatten.uniq!
+today = Time.now.strftime("%Y%m%d")
+all_highlights_filename = "All_Kindle_Highlights #{today}.txt"
+
+	#titles = File.read("My Clippings").scan(/={10}\r\n(.*)\r\n/).flatten.uniq!
+
+# Reset aka wipe the all encompassing highlights file with today's date
+File.write(all_highlights_filename,"")
+
+# Iterate over each title
+# create a file with highlights for each book
+# create an all-encompassing highlight file, grouped by book
+titles.each { |title| 
+	clippings = File.read(kindle_clippings).scan(
+		/#{Regexp.quote(title)}(.*)\r\n(.*)\r\n\r\n(.*)\r\n=======/).flatten
+	highlights = clippings.reject { |clipping| clipping.match(
+		/#{Regexp.quote("- Your Highlight")}/) || clipping.length < 2 }
+	# write file for each book with associated quotes
+	File.write("#{title.strip} #{today}.txt", highlights.join("\n\n"))
+	# write file containing all kindle quotes, grouped by book
+	# writes in a break/line of ='s between books and then the book's title
+	File.write(all_highlights_filename,
+		"="*80  +"\n#{title.strip}\n\n", mode:"a")
+	# writes in all the highlights for the given book
+	File.write(all_highlights_filename,
+		highlights.join("\n\n") + "\n", mode:"a")
+	}
+
+# have 'kindle_clippings' variable able to accept an input - would rather enforce default name

data/kynda_with_comments

@@ -0,0 +1,43 @@
+# Get list of unique book titles from Kindle clippings .txt file
+# Create empty file that will contain highlights from every book
+book_title_regexp = /={10}\r\n(.*)\r\n/
+kindle_clippings = "My Clippings"
+titles = File.read(kindle_clippings).scan(book_title_regexp).flatten.uniq!
+today = Time.now.strftime("%Y%m%d")
+all_highlights_filename = "All_Kindle_Highlights #{today}.txt"
+
+# Reset/wipe the all encompassing highlights file for today's date
+File.write(all_highlights_filename,"")
+
+# Iterate over each title
+# create a file with highlights for each book
+# append (grouped) highlights for each book to the all-encompassing highlight file
+titles.each { |title|	
+	# If loop added because every so often, was getting a book title with
+	# leading spaces - which lead to some repetitive files and highlights
+	# problem child: Amsterdam: A History of the World's Most Liberal City (Shorto, Russell)
+	if title[0] =~ /[\w]/
+		corrected_title = title
+	else
+		corrected_title = title[1..-1]
+	end
+
+	# 'clippings' pulls the text between the line containing the title and author 
+	# and the === break line before the next highlight. This returns one 
+	# match listing the page location and date of the highlight, one match of an 
+	# empty line and one match with the desired highlighted conent (pulled out 
+	# into 'highlights' variable) 
+	clippings = File.read(kindle_clippings).scan(
+		/#{Regexp.quote(corrected_title)}(.*)\r\n(.*)\r\n\r\n(.*)\r\n=======/).flatten
+	highlights = clippings.reject { |clipping| clipping.match(
+		/#{Regexp.quote("- Your Highlight")}/) || clipping.length < 2 }
+	# write file for each book with associated quotes, tagged with today's date
+	File.write("#{corrected_title.strip} #{today}.txt", highlights.join("\n\n"))
+	# write file containing all quotes, grouped by book, tagged w/today's date
+	# writes in a break/line of ='s between books and then the book's title
+	File.write(all_highlights_filename,
+		"="*80  +"\n#{corrected_title.strip}\n\n", mode:"a")
+	# writes in all the highlights for the given book
+	File.write(all_highlights_filename,
+		highlights.join("\n\n") + "\n", mode:"a")
+	}

data/lib/kynda.rb

@@ -0,0 +1,45 @@
+require "kynda/version"
+
+# Get list of unique book titles from Kindle clippings .txt file
+# Create empty file that will contain highlights from every book
+book_title_regexp = /={10}\r\n(.*)\r\n/
+kindle_clippings = "My Clippings"
+titles = File.read(kindle_clippings).scan(book_title_regexp).flatten.uniq!
+today = Time.now.strftime("%Y%m%d")
+all_highlights_filename = "All_Kindle_Highlights #{today}.txt"
+
+# Reset/wipe the all encompassing highlights file for today's date
+File.write(all_highlights_filename,"")
+
+# Iterate over each title
+# create a file with highlights for each book
+# append (grouped) highlights for each book to the all-encompassing highlight file
+titles.each { |title|	
+	# If loop added because every so often, was getting a book title with
+	# leading spaces - which lead to some repetitive files and highlights
+	# problem child: Amsterdam: A History of the World's Most Liberal City (Shorto, Russell)
+	if title[0] =~ /[\w]/
+		corrected_title = title
+	else
+		corrected_title = title[1..-1]
+	end
+
+	# 'clippings' pulls the text between the line containing the title and author 
+	# and the === break line before the next highlight. This returns one 
+	# match listing the page location and date of the highlight, one match of an 
+	# empty line and one match with the desired highlighted conent (pulled out 
+	# into 'highlights' variable) 
+	clippings = File.read(kindle_clippings).scan(
+		/#{Regexp.quote(corrected_title)}(.*)\r\n(.*)\r\n\r\n(.*)\r\n=======/).flatten
+	highlights = clippings.reject { |clipping| clipping.match(
+		/#{Regexp.quote("- Your Highlight")}/) || clipping.length < 2 }
+	# write file for each book with associated quotes, tagged with today's date
+	File.write("#{corrected_title.strip} #{today}.txt", highlights.join("\n\n"))
+	# write file containing all quotes, grouped by book, tagged w/today's date
+	# writes in a break/line of ='s between books and then the book's title
+	File.write(all_highlights_filename,
+		"="*80  +"\n#{corrected_title.strip}\n\n", mode:"a")
+	# writes in all the highlights for the given book
+	File.write(all_highlights_filename,
+		highlights.join("\n\n") + "\n", mode:"a")
+	}

data/lib/kynda/version.rb

@@ -0,0 +1,3 @@
+module Kynda
+  VERSION = "0.1.0"
+end

data/rubyscript

@@ -0,0 +1,50 @@
+#split on title line
+#group by titles
+#format
+
+# File.read eg:  text = File.read("clippings.txt")
+
+# Regex to get titles: /==========(.*)\n(.*)/
+
+titles = File.read("shortclippings.txt").scan(/={10}\n(.*)\n/).flatten
+
+# remove duplicates
+	#.inject combo with .include?
+
+titlelist = titles.uniq
+  #with Regex, first line is missing add in with something like: 
+  #>> titlelist.include?(File.readlines("shortclippings.txt")[0])   BUT need to get that other leading space off first then to avoid doubling down on the Amsterdam book
+
+## skip this for now but the Amsterdam title example has a leading space, can't get it off without making things worse
+titlelist.map {|t| t.strip}
+titlelist.map {|t| t.to_s.strip}
+
+# each loop...for each title, find all the highlights and output (combined file and individual books files?)
+##look for match on title[n](.*)={10}/m
+
+puts File.read("shortclippings.txt").scan(/#{Regexp.quote(titlelist[0])}/)
+##the code on the next line below will return everything for a given title, that is below
+##the title line and above the ========== line (includes the your highlight page/date/time line)
+##put all these into a hash/array organized by book and then work on cleaning them up?
+puts File.read("shortclippings.txt").scan(/#{Regexp.quote(titlelist[0])}(.*)\n(.*)\n\n(.*)\n=======/)
+#looks like can add .flatten to this and then want every third element as a quote [0],[2],[5],etc.
+# NOTE - using titlelist[0] will only give us clippings for the first book, how to iterate this?
+
+##OLD THOUGHTS
+# get i for 0 ...quote length, then pull out the modul0 3 ones (would need to add in [0] 
+# 	or unshift a few placeholder elements to the front of the array...iterate for each quotes[i] value?)
+#clippings.unshift #one time means [3], [6], etc will pull out actual clippings now.
+
+#could also use reject to just get true 'clippings' in
+#a variable and then use File.write to add the to the file (apppend
+#if dont' want to overwrite
+##### REVERSE CLIPPING AND QUOTE TERMINOLOGY #####
+highlights = clippings.reject { |clipping| clipping.match(/#{Regexp.quote("- Your Highlight")}/) || clipping.length < 2 }
+
+##highlights variable will now hold all 'true' highlights and no empty strings 
+##or location time values
+
+#output to file works, may want to insert empty row between each
+#need to include title/rename file based on title
+#may also want to append into "all highlights" file
+File.write('file_title_stub.txt', highlights.join("\n"))

data/shortclippings.txt

@@ -0,0 +1,40 @@
+Amsterdam: A History of the World's Most Liberal City (Shorto, Russell)
+- Your Highlight on page 15 | Location 200-201 | Added on Thursday, January 26, 2017 6:01:54 AM
+
+gedogen
+==========
+Amsterdam: A History of the World's Most Liberal City (Shorto, Russell)
+- Your Highlight on page 15 | Location 200-201 | Added on Thursday, January 26, 2017 6:02:03 AM
+
+gedogen, which means “technically illegal but officially tolerated.”
+==========
+Amsterdam: A History of the World's Most Liberal City (Shorto, Russell)
+- Your Highlight on page 16 | Location 213-214 | Added on Thursday, January 26, 2017 6:04:20 AM
+
+Amsterdam is famous for one thing (besides canals, and cannabis cafés, and prostitutes): the tattered, ancient, much-misunderstood word liberalism. Amsterdam is, by most accounts, the most liberal place on earth.
+==========
+Amsterdam: A History of the World's Most Liberal City (Shorto, Russell)
+- Your Highlight on page 21 | Location 299-300 | Added on Thursday, January 26, 2017 6:21:08 AM
+
+Multiculturalism proved to be a failure. It was leading not to a mixed society but to a multiplicity of ghettoized communities living next to but cut off from one another: the very opposite of a “society.”
+==========
+Amsterdam: A History of the World's Most Liberal City (Shorto, Russell)
+- Your Highlight on page 24 | Location 336-338 | Added on Thursday, January 26, 2017 6:26:11 AM
+
+“We were against authority and ownership. Buying real estate was for patsers.” (The word doesn’t translate well, but “establishment assholes” gets close.)
+==========
+Deep Work (Cal Newport)
+- Your Highlight on Location 1391-1393 | Added on Wednesday, July 1, 2020 7:09:58 AM
+
+In all of these examples, it’s not just the change of environment or seeking of quiet that enables more depth. The dominant force is the psychology of committing so seriously to the task at hand.
+==========
+Deep Work (Cal Newport)
+- Your Highlight on Location 1394-1395 | Added on Wednesday, July 1, 2020 7:10:07 AM
+
+These gestures push your deep goal to a level of mental priority that helps unlock the needed mental resources. Sometimes to go deep, you must first go big.
+==========
+Egypt on the Brink: From the Rise of Nasser to the Fall of Mubarak (Osman, Tarek)
+- Your Highlight on page 41 | Location 700-705 | Added on Thursday, August 2, 2018 6:27:09 PM
+
+Free trade, open markets and vast opportunities for wealth creation turned Egypt into a regional commercial and trading destination. One consequence was a flow of immigrants; the number of foreigners in the country rose from 10,000 in the 1840s (the first relatively reliable census) to around 90,000 in the 1880s, and more than 1.5 million by the 1930s. The major immigrant populations included Greeks, Jews and Armenians.24 The ensuing cosmopolitanism infused Egyptian society with tolerance, open-mindedness and glamour  adding to the country's ‘seductive soft power’, and gave the liberal experiment a certain international appeal.
+==========