sakuric 2.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/CHANGELOG +13 -0
- data/Gemfile +22 -0
- data/Gemfile.lock +60 -0
- data/HISTORY.yml +13 -0
- data/INSTALL +5 -0
- data/LICENSE +1 -0
- data/Makefile +19 -0
- data/Manifest +405 -0
- data/README.md +72 -0
- data/Rakefile +4 -0
- data/TODO +3 -0
- data/VERSION +1 -0
- data/bashrc +30 -0
- data/bashrc.d/00-functions.include +7 -0
- data/bashrc.d/01-sakura_checks.include +15 -0
- data/bashrc.d/02-uname.include +11 -0
- data/bashrc.d/80-ruby.include +4 -0
- data/bashrc.d/99-dirs.include +11 -0
- data/bashrc.d/aliases.include +20 -0
- data/bashrc.d/common.include +7 -0
- data/bashrc.d/local.d/README +1 -0
- data/bashrc.d/local.d/nothing.include +0 -0
- data/bashrc.d/uname.d/Darwin +6 -0
- data/bashrc.d/uname.d/Linux +0 -0
- data/bashrc.local.sample +11 -0
- data/bin/10 +7 -0
- data/bin/10times +7 -0
- data/bin/1every +22 -0
- data/bin/1everyN +22 -0
- data/bin/1suN +22 -0
- data/bin/2 +5 -0
- data/bin/act +24 -0
- data/bin/amarelo +2 -0
- data/bin/arancio +4 -0
- data/bin/arcobaleno +10 -0
- data/bin/bianco +3 -0
- data/bin/black +2 -0
- data/bin/brew-install +3 -0
- data/bin/cache +50 -0
- data/bin/cache3 +50 -0
- data/bin/cheorae +4 -0
- data/bin/colorastdouterr +37 -0
- data/bin/cp2desktop +3 -0
- data/bin/create-git-repo.sh +24 -0
- data/bin/csoe +37 -0
- data/bin/data +3 -0
- data/bin/derive +90 -0
- data/bin/dimmi +31 -0
- data/bin/dimmiora +4 -0
- data/bin/doppioni +48 -0
- data/bin/du- +9 -0
- data/bin/duplicates +48 -0
- data/bin/echodo +7 -0
- data/bin/ensure_line_present +82 -0
- data/bin/epoch2date +5 -0
- data/bin/fanout +100 -0
- data/bin/find-broken-symlink +26 -0
- data/bin/find-duplicates +48 -0
- data/bin/find-special-chars-within-filenames +12 -0
- data/bin/fix-terminal +5 -0
- data/bin/gce-create-centos-instance +5 -0
- data/bin/gce-create-custom-image +8 -0
- data/bin/gce-curl-get-instances-for-project +7 -0
- data/bin/gce-get-token +3 -0
- data/bin/gce-getproject-metadata +35 -0
- data/bin/gce-public-ip +4 -0
- data/bin/gcutil-fetch +36 -0
- data/bin/giallo +2 -0
- data/bin/git-clone-in-non-empty-dir +21 -0
- data/bin/git-ignore +3 -0
- data/bin/git-repos +98 -0
- data/bin/git-repos.py +99 -0
- data/bin/gitinfo +2 -0
- data/bin/google-spreadsheet-cat +40 -0
- data/bin/google_ips +5 -0
- data/bin/gray +3 -0
- data/bin/green +2 -0
- data/bin/grigio +3 -0
- data/bin/gsutil-debug-bucket +12 -0
- data/bin/gsutil-get-oauth2-token +4 -0
- data/bin/gugol-image +74 -0
- data/bin/hamachi-ubuntu-install +17 -0
- data/bin/head-1^-1 +5 -0
- data/bin/ifdownup +13 -0
- data/bin/immature/ok2 +25 -0
- data/bin/immature/twice-still-broken +9 -0
- data/bin/install-gic-repo +13 -0
- data/bin/install-sakura-on-debian-latest +6 -0
- data/bin/iphoto_big_files.sh +37 -0
- data/bin/itunes +140 -0
- data/bin/jason-xpath.rb +54 -0
- data/bin/jsawk +1128 -0
- data/bin/json-xpath.rb +54 -0
- data/bin/keepup +13 -0
- data/bin/lsof-top10 +3 -0
- data/bin/minicook +218 -0
- data/bin/mv2. +2 -0
- data/bin/mv2desktop +2 -0
- data/bin/mv2dropbox +2 -0
- data/bin/mv2here +2 -0
- data/bin/mvto +39 -0
- data/bin/nero +2 -0
- data/bin/never_as_root +18 -0
- data/bin/not +16 -0
- data/bin/ok +42 -0
- data/bin/orange +4 -0
- data/bin/pink +2 -0
- data/bin/polygen-dell +7 -0
- data/bin/polygen-heanet +6 -0
- data/bin/polygen-labs +4 -0
- data/bin/print_key_val +4 -0
- data/bin/rainbow +10 -0
- data/bin/recipe +19 -0
- data/bin/red +2 -0
- data/bin/remember-command +140 -0
- data/bin/remember-command.rb +140 -0
- data/bin/richelp +99 -0
- data/bin/ricorda-comando +140 -0
- data/bin/rosa +2 -0
- data/bin/rosso +2 -0
- data/bin/run-parts.sh +50 -0
- data/bin/sakura-check-version +48 -0
- data/bin/sakura-init.DRAFT +5 -0
- data/bin/sakura-version +2 -0
- data/bin/sano +5 -0
- data/bin/sbianca +3 -0
- data/bin/sblua +2 -0
- data/bin/setterm-reset +5 -0
- data/bin/sgialla +2 -0
- data/bin/sgrigia +2 -0
- data/bin/split +37 -0
- data/bin/split.rb +37 -0
- data/bin/srosa +1 -0
- data/bin/srossa +2 -0
- data/bin/strip-white-spaces +2 -0
- data/bin/stty-sane +5 -0
- data/bin/sudo^-1 +18 -0
- data/bin/sverda +6 -0
- data/bin/swift-saio-install.sh +336 -0
- data/bin/synergyc +0 -0
- data/bin/synergys +0 -0
- data/bin/syracuse.pl +29 -0
- data/bin/tail-n+2 +5 -0
- data/bin/tellme +31 -0
- data/bin/tellme-time +4 -0
- data/bin/timeout3 +91 -0
- data/bin/top5 +4 -0
- data/bin/trim +4 -0
- data/bin/trova-broken-symlink +26 -0
- data/bin/twice +5 -0
- data/bin/twice.rb +5 -0
- data/bin/ubuntu-install-hamachi64 +17 -0
- data/bin/ultimo +5 -0
- data/bin/upload.py +2380 -0
- data/bin/usleep +0 -0
- data/bin/varia +90 -0
- data/bin/verde +2 -0
- data/bin/vermelho +2 -0
- data/bin/vim-whereveritis +5 -0
- data/bin/viola +1 -0
- data/bin/virsh-list-all-vms.py +21 -0
- data/bin/virtualbox-manage +41 -0
- data/bin/weekend +12 -0
- data/bin/whattimeisit +4 -0
- data/bin/whereis-ip +5 -0
- data/bin/white +3 -0
- data/bin/whitefy +3 -0
- data/bin/yellow +2 -0
- data/bin/yellowfy +2 -0
- data/bin/zombies +25 -0
- data/docz/polygen/dell.grm +100 -0
- data/docz/polygen/frati.grm +85 -0
- data/docz/polygen/heanet.grm +54 -0
- data/docz/richelp/gcompute.yml +52 -0
- data/docz/richelp/gcutil.yml +52 -0
- data/docz/richelp/gmail.yml +2 -0
- data/docz/richelp/ruby.yml +81 -0
- data/docz/richelp/sakura.yml +20 -0
- data/docz/richelp/sql.yml +12 -0
- data/docz/richelp/ubuntu.yml +22 -0
- data/docz/richelp/yaml.yml +39 -0
- data/docz/richelp/yml.yml +39 -0
- data/etc/sakura/devel.yml +36 -0
- data/etc/synergy.conf.example +37 -0
- data/images/color-sample.png +0 -0
- data/images/google.jpg +0 -0
- data/images/icons/toilet.png +0 -0
- data/images/sakura.jpg +0 -0
- data/images/sakura2.jpg +0 -0
- data/init.rb +5 -0
- data/lib/classes/arrays.rb +246 -0
- data/lib/classes/debug_ric.rb +196 -0
- data/lib/classes/fake_data.rb +8 -0
- data/lib/classes/ric_debug.rb +0 -0
- data/lib/classes/ricconf.rb +110 -0
- data/lib/classes/richelp.rb +70 -0
- data/lib/classes/strings.rb +284 -0
- data/lib/facter/google_compute_engine.rb +10 -0
- data/lib/recipes/20120411-puppet-rump-palladius.recipe +2 -0
- data/lib/recipes/20120726-skype.ubuntu.recipe +16 -0
- data/lib/recipes/20130206-puppet3.ubuntu.recipe +32 -0
- data/lib/recipes/20130206-redis.linux.recipe +17 -0
- data/lib/recipes/20130208-vanilla.linux.recipe +37 -0
- data/lib/recipes/Makefile +4 -0
- data/lib/recipes/README.md +29 -0
- data/lib/recipes/scripts/redis.sh +14 -0
- data/lib/ric.rb +156 -0
- data/lib/ric_colors.rb +313 -0
- data/lib/ric_strings.rb +285 -0
- data/lib/sonice-players/itunes_mac.rb +43 -0
- data/lib/sonice-players/itunes_win.rb +47 -0
- data/lib/sonice-players/mpd.rb +45 -0
- data/lib/sonice-players/rhythmbox.rb +38 -0
- data/profile +1 -0
- data/sakuric.gemspec +40 -0
- data/sbin/install-sakura-on-debian +25 -0
- data/sbin/make-install.sh +10 -0
- data/sbin/mate-createsymlink +4 -0
- data/sounds/0.ogg +0 -0
- data/sounds/0.wav +0 -0
- data/sounds/01.ogg +0 -0
- data/sounds/02.ogg +0 -0
- data/sounds/03.ogg +0 -0
- data/sounds/08.ogg +0 -0
- data/sounds/09.ogg +0 -0
- data/sounds/1.ogg +0 -0
- data/sounds/1.wav +0 -0
- data/sounds/10.ogg +0 -0
- data/sounds/100.ogg +0 -0
- data/sounds/11.ogg +0 -0
- data/sounds/12.ogg +0 -0
- data/sounds/13.ogg +0 -0
- data/sounds/14.ogg +0 -0
- data/sounds/15.ogg +0 -0
- data/sounds/16.ogg +0 -0
- data/sounds/17.ogg +0 -0
- data/sounds/18.ogg +0 -0
- data/sounds/19.ogg +0 -0
- data/sounds/2.ogg +0 -0
- data/sounds/2.wav +0 -0
- data/sounds/20.ogg +0 -0
- data/sounds/21.ogg +0 -0
- data/sounds/22.ogg +0 -0
- data/sounds/23.ogg +0 -0
- data/sounds/24.ogg +0 -0
- data/sounds/26.ogg +0 -0
- data/sounds/3.ogg +0 -0
- data/sounds/3.wav +0 -0
- data/sounds/30.ogg +0 -0
- data/sounds/34.ogg +0 -0
- data/sounds/4.ogg +0 -0
- data/sounds/4.wav +0 -0
- data/sounds/40.ogg +0 -0
- data/sounds/42.ogg +0 -0
- data/sounds/45.ogg +0 -0
- data/sounds/47.ogg +0 -0
- data/sounds/5.ogg +0 -0
- data/sounds/5.wav +0 -0
- data/sounds/50.ogg +0 -0
- data/sounds/6.ogg +0 -0
- data/sounds/6.wav +0 -0
- data/sounds/60.ogg +0 -0
- data/sounds/7.ogg +0 -0
- data/sounds/7.wav +0 -0
- data/sounds/70.ogg +0 -0
- data/sounds/8.ogg +0 -0
- data/sounds/8.wav +0 -0
- data/sounds/80.ogg +0 -0
- data/sounds/9.ogg +0 -0
- data/sounds/9.wav +0 -0
- data/sounds/90.ogg +0 -0
- data/sounds/a.ogg +0 -0
- data/sounds/aaahhh.ogg +0 -0
- data/sounds/abologna.ogg +0 -0
- data/sounds/adargenta.ogg +0 -0
- data/sounds/alzatiecammina.ogg +0 -0
- data/sounds/apache.ogg +0 -0
- data/sounds/attenzione.ogg +0 -0
- data/sounds/awesome.m4a +0 -0
- data/sounds/bazinga.mp3 +0 -0
- data/sounds/bazinga.ogg +0 -0
- data/sounds/bela regaz.wav +0 -0
- data/sounds/benvenuti.wav +0 -0
- data/sounds/benvenuti2.wav +0 -0
- data/sounds/bergonz.ogg +0 -0
- data/sounds/boh.ogg +0 -0
- data/sounds/buahah.ogg +0 -0
- data/sounds/buonasera.ogg +0 -0
- data/sounds/buongiorno.ogg +0 -0
- data/sounds/burp.ogg +0 -0
- data/sounds/carlo.ogg +0 -0
- data/sounds/cena.ogg +0 -0
- data/sounds/che figata.m4a +0 -0
- data/sounds/che figlio di puttana.m4a +0 -0
- data/sounds/che.ogg +0 -0
- data/sounds/ciao.ogg +0 -0
- data/sounds/ciao.wav +0 -0
- data/sounds/da.ogg +0 -0
- data/sounds/davide.ogg +0 -0
- data/sounds/demone.ogg +0 -0
- data/sounds/dhcp.ogg +0 -0
- data/sounds/dinuovo.ogg +0 -0
- data/sounds/dns.ogg +0 -0
- data/sounds/dopo.ogg +0 -0
- data/sounds/e'.ogg +0 -0
- data/sounds/eattivo.ogg +0 -0
- data/sounds/ee.ogg +0 -0
- data/sounds/ehi.ogg +0 -0
- data/sounds/eoragiu.ogg +0 -0
- data/sounds/eorasu.ogg +0 -0
- data/sounds/eripartita.ogg +0 -0
- data/sounds/estato.ogg +0 -0
- data/sounds/estatoriavviato.ogg +0 -0
- data/sounds/fratro.ogg +0 -0
- data/sounds/frozenbubble.ogg +0 -0
- data/sounds/funziona.ogg +0 -0
- data/sounds/go raibh mile maith agaibh.m4a +0 -0
- data/sounds/grazie.ogg +0 -0
- data/sounds/h1.ogg +0 -0
- data/sounds/hainuovaposta.ogg +0 -0
- data/sounds/hogiamangiato.ogg +0 -0
- data/sounds/host.ogg +0 -0
- data/sounds/il.ogg +0 -0
- data/sounds/ilcomputer.ogg +0 -0
- data/sounds/ilnumerodaleiselezionatoe.ogg +0 -0
- data/sounds/ilprocesso.ogg +0 -0
- data/sounds/ilprogramma.ogg +0 -0
- data/sounds/ilservizio.ogg +0 -0
- data/sounds/imap.ogg +0 -0
- data/sounds/imieisuperpoterimidiconoche.ogg +0 -0
- data/sounds/infunzione.ogg +0 -0
- data/sounds/inpunto.ogg +0 -0
- data/sounds/ipmon.ogg +0 -0
- data/sounds/laconnessioneadinternet.ogg +0 -0
- data/sounds/lastampante.ogg +0 -0
- data/sounds/lazzaron.ogg +0 -0
- data/sounds/ldap.ogg +0 -0
- data/sounds/linterfaccia.ogg +0 -0
- data/sounds/lucilla.ogg +0 -0
- data/sounds/ma vaffanculo.m4a +0 -0
- data/sounds/majjal.wav +0 -0
- data/sounds/mamma.ogg +0 -0
- data/sounds/mauro.ogg +0 -0
- data/sounds/max.ogg +0 -0
- data/sounds/meerda.wav +0 -0
- data/sounds/meno.ogg +0 -0
- data/sounds/merda clo.wav +0 -0
- data/sounds/mezza.ogg +0 -0
- data/sounds/mila.ogg +0 -0
- data/sounds/mille.ogg +0 -0
- data/sounds/minuti.ogg +0 -0
- data/sounds/named.ogg +0 -0
- data/sounds/no.ogg +0 -0
- data/sounds/nohofame.ogg +0 -0
- data/sounds/nonfunziona.ogg +0 -0
- data/sounds/nonriescoatrovare.ogg +0 -0
- data/sounds/nonsarebbeoradilavorare.ogg +0 -0
- data/sounds/nonstalavorando.ogg +0 -0
- data/sounds/numero.wav +0 -0
- data/sounds/ooohhh.ogg +0 -0
- data/sounds/pinger.ogg +0 -0
- data/sounds/pocofa.ogg +0 -0
- data/sounds/pop3.ogg +0 -0
- data/sounds/porca puttana.m4a +0 -0
- data/sounds/porcapupazza.ogg +0 -0
- data/sounds/ppp.ogg +0 -0
- data/sounds/pranzo.ogg +0 -0
- data/sounds/prego.ogg +0 -0
- data/sounds/prima.ogg +0 -0
- data/sounds/qualcosamidice.ogg +0 -0
- data/sounds/raggiungibile.ogg +0 -0
- data/sounds/riccardo.ogg +0 -0
- data/sounds/salsa ridge.wav +0 -0
- data/sounds/scusa.ogg +0 -0
- data/sounds/seriously man.m4a +0 -0
- data/sounds/si.ogg +0 -0
- data/sounds/sito porno con sonoro.wav +0 -0
- data/sounds/socmel.wav +0 -0
- data/sounds/sono.ogg +0 -0
- data/sounds/sonoacasa.ogg +0 -0
- data/sounds/sonole.ogg +0 -0
- data/sounds/sputo.ogg +0 -0
- data/sounds/ssh.ogg +0 -0
- data/sounds/stagiocando.ogg +0 -0
- data/sounds/statospento.ogg +0 -0
- data/sounds/stopaiodipalle.ogg +0 -0
- data/sounds/thesedicks.wav +0 -0
- data/sounds/tornatosu.ogg +0 -0
- data/sounds/trapoco.ogg +0 -0
- data/sounds/trequarti.ogg +0 -0
- data/sounds/tutti.ogg +0 -0
- data/sounds/una.ogg +0 -0
- data/sounds/unquarto.ogg +0 -0
- data/sounds/virgola.ogg +0 -0
- data/sources/c/usleep.c +50 -0
- data/spec/my_first_spec.rb +6 -0
- data/tasks/sakuric-gem.rake +21 -0
- data/tasks/testing.rake +15 -0
- data/templates/bashrc.inject +10 -0
- data/templates/happy_christmas.ascii_art +5 -0
- data/test/roodi.sh +10 -0
- data/test/test_mini_failing.rb +3 -0
- data/test/test_mini_ok.rb +3 -0
- data/test/test_ruby_syntax.sh +12 -0
- data/third-party/jsawk/README.markdown +437 -0
- data/third-party/jsawk/jsawk +1128 -0
- metadata +835 -0
data/sounds/virgola.ogg
ADDED
|
Binary file
|
data/sources/c/usleep.c
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
#include<stdio.h>
|
|
2
|
+
#include<time.h>
|
|
3
|
+
#include<signal.h>
|
|
4
|
+
|
|
5
|
+
// found it here: http://cc.byexamples.com/2007/05/25/nanosleep-is-better-than-sleep-and-usleep/
|
|
6
|
+
|
|
7
|
+
void sigfunc(int sig_no)
|
|
8
|
+
{
|
|
9
|
+
|
|
10
|
+
}
|
|
11
|
+
|
|
12
|
+
int __nsleep(const struct timespec *req, struct timespec *rem)
|
|
13
|
+
{
|
|
14
|
+
struct timespec temp_rem;
|
|
15
|
+
if(nanosleep(req,rem)==-1)
|
|
16
|
+
__nsleep(rem,&temp_rem);
|
|
17
|
+
else
|
|
18
|
+
return 1;
|
|
19
|
+
}
|
|
20
|
+
|
|
21
|
+
int msleep(unsigned long milisec)
|
|
22
|
+
{
|
|
23
|
+
struct timespec req={0},rem={0};
|
|
24
|
+
time_t sec=(int)(milisec/1000);
|
|
25
|
+
milisec=milisec-(sec*1000);
|
|
26
|
+
req.tv_sec=sec;
|
|
27
|
+
req.tv_nsec=milisec*1000000L;
|
|
28
|
+
__nsleep(&req,&rem);
|
|
29
|
+
return 1;
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
int main()
|
|
33
|
+
{
|
|
34
|
+
struct sigaction sa={0};
|
|
35
|
+
sa.sa_handler=&sigfunc;
|
|
36
|
+
sigaction(SIGINT, &sa,NULL);
|
|
37
|
+
|
|
38
|
+
int a=0;
|
|
39
|
+
scanf("%d",&a);
|
|
40
|
+
|
|
41
|
+
for (;;)
|
|
42
|
+
{
|
|
43
|
+
printf("testing...\n");
|
|
44
|
+
if (a==1)
|
|
45
|
+
msleep(1000);
|
|
46
|
+
else
|
|
47
|
+
usleep(1000000);
|
|
48
|
+
}
|
|
49
|
+
return 1;
|
|
50
|
+
}
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
|
|
2
|
+
#################
|
|
3
|
+
# Deploy the gem 'sakuric'
|
|
4
|
+
|
|
5
|
+
version = File.read( 'VERSION' ) rescue "0.0.42_sakbug"
|
|
6
|
+
|
|
7
|
+
Echoe.new('sakuric', version ) do |p|
|
|
8
|
+
p.description = "My SAKURA gem with various utilities. This is my swiss-army knife for Linux and Mac. See README.md for amazing examples"
|
|
9
|
+
p.url = "http://github.com/palladius/sakura"
|
|
10
|
+
p.author = "Riccardo Carlesso"
|
|
11
|
+
p.email = "['p','ll','diusbonton].join('a') @ gmail.com"
|
|
12
|
+
p.ignore_pattern = [
|
|
13
|
+
"tmp/*",
|
|
14
|
+
"tmp/*", #"tmp/*/*", "tmp/*/*/*",
|
|
15
|
+
"private/*",
|
|
16
|
+
".noheroku",
|
|
17
|
+
'.travis.yml',
|
|
18
|
+
]
|
|
19
|
+
p.development_dependencies = [ 'ric','echoe' ]
|
|
20
|
+
p.runtime_dependencies = [ 'ric' ]
|
|
21
|
+
end
|
data/tasks/testing.rake
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
|
|
2
|
+
desc "Run tests hypothetically"
|
|
3
|
+
task :default => :rictest
|
|
4
|
+
|
|
5
|
+
namespace :rictest do
|
|
6
|
+
desc "Run mini tests"
|
|
7
|
+
puts 'Rake test being executed...'
|
|
8
|
+
Dir['test/*.rb'].each do |file|
|
|
9
|
+
system "ruby #{file}"
|
|
10
|
+
end
|
|
11
|
+
Dir['test/*.sh'].each do |file|
|
|
12
|
+
system "#{file}"
|
|
13
|
+
end
|
|
14
|
+
end
|
|
15
|
+
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
|
|
2
|
+
#######################################################################
|
|
3
|
+
#palladius/sakura BEGIN
|
|
4
|
+
# Add me to ~/.bashrc, this is inside the `bashrc.inject`
|
|
5
|
+
export SAKURADIR=~/git/sakura/ # or whatever
|
|
6
|
+
if [ -f "$SAKURADIR/bashrc" ]; then
|
|
7
|
+
. "$SAKURADIR/bashrc"
|
|
8
|
+
fi
|
|
9
|
+
#/palladius/sakura END
|
|
10
|
+
#######################################################################
|
data/test/roodi.sh
ADDED
|
@@ -0,0 +1,437 @@
|
|
|
1
|
+
Quick Start
|
|
2
|
+
===========
|
|
3
|
+
|
|
4
|
+
[Updated underscore.js to v1.3.1.](http://documentcloud.github.com/underscore/)
|
|
5
|
+
|
|
6
|
+
Jsawk is like awk, but for JSON. You work with an array of JSON objects
|
|
7
|
+
read from stdin, filter them using JavaScript to produce a results array
|
|
8
|
+
that is printed to stdout. You can use this as a filter to manipulate data
|
|
9
|
+
from a REST JSON web service, for example, in a shell script. Also, you can
|
|
10
|
+
suppress JSON output and use the built-in printing functions to translate
|
|
11
|
+
your JSON input to other formats and send that to stdout, to be piped to
|
|
12
|
+
other processes. You can load JavaScript libraries on the command line to
|
|
13
|
+
increase your processing power, and other things.
|
|
14
|
+
|
|
15
|
+
Setup
|
|
16
|
+
-----
|
|
17
|
+
|
|
18
|
+
[This is a great blog post on setup and basic use of jsawk and resty, thanks
|
|
19
|
+
to @johnattebury.](http://johnattebury.com/blog/2011/06/spidermonkey-jsawk-resty-on-snow-leopard/)
|
|
20
|
+
|
|
21
|
+
You need to have the `js` interpreter installed. On a Debian system you can
|
|
22
|
+
install it with the following command:
|
|
23
|
+
|
|
24
|
+
apt-get install spidermonkey-bin
|
|
25
|
+
|
|
26
|
+
Or you can [build it from source](http://www.mozilla.org/js/spidermonkey/).
|
|
27
|
+
Ready? Go.
|
|
28
|
+
|
|
29
|
+
Install
|
|
30
|
+
-------
|
|
31
|
+
|
|
32
|
+
First, get the jsawk script:
|
|
33
|
+
|
|
34
|
+
curl -L http://github.com/micha/jsawk/raw/master/jsawk > jsawk
|
|
35
|
+
|
|
36
|
+
Then make it executable and put it somewhere in your path:
|
|
37
|
+
|
|
38
|
+
chmod 755 jsawk && mv jsawk ~/bin/
|
|
39
|
+
|
|
40
|
+
Use
|
|
41
|
+
---
|
|
42
|
+
|
|
43
|
+
Now you can do some stuff with JSON data. Here's an example using data from
|
|
44
|
+
a REST service that serves JSON (we use [resty](http://github.com/micha/resty)
|
|
45
|
+
to do the HTTP requests):
|
|
46
|
+
|
|
47
|
+
resty http://example.com:8080/data*.json
|
|
48
|
+
GET /people/47 | jsawk 'this.favoriteColor = "blue"' | PUT /people/47
|
|
49
|
+
|
|
50
|
+
This would do a `GET` request on the resource `/data/people/47.json`, which
|
|
51
|
+
would result in a JSON object. Then jsawk takes the JSON via stdin and for
|
|
52
|
+
each JSON object it runs the little snippet of JavaScript, setting the
|
|
53
|
+
`favoriteColor` property to `"blue"`, in this case. The modified JSON is then
|
|
54
|
+
output via stdout to `resty` again, which does the `PUT` request to update
|
|
55
|
+
the resource.
|
|
56
|
+
|
|
57
|
+
Usage
|
|
58
|
+
=====
|
|
59
|
+
|
|
60
|
+
jsawk [OPTIONS] [SCRIPT]
|
|
61
|
+
|
|
62
|
+
OPTIONS
|
|
63
|
+
-------
|
|
64
|
+
|
|
65
|
+
-h
|
|
66
|
+
Print short help page and exit.
|
|
67
|
+
|
|
68
|
+
-n
|
|
69
|
+
Suppress printing of JSON result set.
|
|
70
|
+
|
|
71
|
+
-q <query>
|
|
72
|
+
Filter JSON through the specified JSONQuery query. If multiple
|
|
73
|
+
'-q' options are specified then each query will be performed in
|
|
74
|
+
turn, in the order in which they appeared on the command line.
|
|
75
|
+
|
|
76
|
+
-v <name=value>
|
|
77
|
+
Set global variable `name` to `value` in the script environment.
|
|
78
|
+
|
|
79
|
+
-f <file>
|
|
80
|
+
Load and run the specified JavaScript file prior to processing
|
|
81
|
+
JSON. This option can be specified multiple times to load multiple
|
|
82
|
+
JavaScript libraries.
|
|
83
|
+
|
|
84
|
+
-b <script> | -a <script>
|
|
85
|
+
Run the specified snippet of JavaScript before (-b) or after (-a)
|
|
86
|
+
processing JSON input. The `this` object is set to the whole JSON
|
|
87
|
+
array or object. This is used to preprocess (-b) or postprocess
|
|
88
|
+
(-a) the JSON array before or after the main script is applied.
|
|
89
|
+
This option can be specified multiple times to define multiple
|
|
90
|
+
before/after scripts, which will be applied in the order they
|
|
91
|
+
appeared on the command line.
|
|
92
|
+
|
|
93
|
+
SCRIPT
|
|
94
|
+
------
|
|
95
|
+
|
|
96
|
+
This is a snippet of JavaScript that will be run on each element
|
|
97
|
+
of the input array, if input is a JSON array, or on the object if
|
|
98
|
+
it's an object. For each iteration, the `this` object is set to the
|
|
99
|
+
current element.
|
|
100
|
+
|
|
101
|
+
Jsawk Scripting
|
|
102
|
+
===============
|
|
103
|
+
|
|
104
|
+
Jsawk is intended to serve the purpose that is served by `awk` in the shell
|
|
105
|
+
environment, but instead of working with words and lines of text, it works
|
|
106
|
+
with JavaScript objects and arrays of objects.
|
|
107
|
+
|
|
108
|
+
In awk, a text file is split into an array of "records", each of which being
|
|
109
|
+
an array of "fields". The awk script that is specified on the command line is
|
|
110
|
+
run once for each record in the array, with the `$1`, `$2`, etc. variables
|
|
111
|
+
set to the various fields in the record. The awk script can set variables,
|
|
112
|
+
perform calculations, do various text-munging things, and print output. This
|
|
113
|
+
printing capablity makes awk into a filter, taking text input, transforming
|
|
114
|
+
it record by record, printing out the resulting modified records at the end.
|
|
115
|
+
|
|
116
|
+
Jsawk is similar, but in jsawk records are elements of the JSON input array
|
|
117
|
+
(if the input was a single object then there is a single record consisting
|
|
118
|
+
of that object). The jsawk script is run once for each record object, with
|
|
119
|
+
the `this` object set to the current record. So here the properties of the
|
|
120
|
+
record object are equivalent to the `$1`, `$2`, etc. in awk. The jsawk
|
|
121
|
+
script can then modify the record, perform calculations, do things. However,
|
|
122
|
+
instead of printing the modified record, the modified record is `return`ed.
|
|
123
|
+
At then end, if the `-n` option was not specified, the resulting array is
|
|
124
|
+
printed as JSON to stdout.
|
|
125
|
+
|
|
126
|
+
Jsawk JavaScript Environment
|
|
127
|
+
----------------------------
|
|
128
|
+
|
|
129
|
+
Jsawk uses the Spidermonkey JavaScript interpreter, so you have access to all
|
|
130
|
+
of the Spidermonkey functions and whatnot. Additionally, the following
|
|
131
|
+
functions and properties are available from within a jsawk script:
|
|
132
|
+
|
|
133
|
+
PROPERTIES
|
|
134
|
+
----------
|
|
135
|
+
|
|
136
|
+
window
|
|
137
|
+
The global object.
|
|
138
|
+
|
|
139
|
+
IS
|
|
140
|
+
The input set.
|
|
141
|
+
|
|
142
|
+
RS
|
|
143
|
+
The result set.
|
|
144
|
+
|
|
145
|
+
_ The underscore.js object.
|
|
146
|
+
|
|
147
|
+
$_
|
|
148
|
+
The current record index (corresponding to the index of the
|
|
149
|
+
element in the IS array).
|
|
150
|
+
|
|
151
|
+
$$
|
|
152
|
+
The current record object (global variable corresponding to the
|
|
153
|
+
`this` object in the script scope).
|
|
154
|
+
|
|
155
|
+
METHODS
|
|
156
|
+
-------
|
|
157
|
+
|
|
158
|
+
forEach(array, string)
|
|
159
|
+
Compiles 'string' into a function and iterates over the 'array',
|
|
160
|
+
running the function once for each element. The function has
|
|
161
|
+
access to the special variables 'index' and 'item' which are,
|
|
162
|
+
respectively, the array index and the array element. The 'this'
|
|
163
|
+
object is set to the array element each time the function runs.
|
|
164
|
+
|
|
165
|
+
params: Array array (array to iterate over)
|
|
166
|
+
String string (the function source)
|
|
167
|
+
return: void
|
|
168
|
+
|
|
169
|
+
get()
|
|
170
|
+
Get the next record from the input set. This will prevent jsawk
|
|
171
|
+
from iterating over that record.
|
|
172
|
+
|
|
173
|
+
params: void
|
|
174
|
+
return: Object|Array|Number|String (the next input record)
|
|
175
|
+
|
|
176
|
+
put(record)
|
|
177
|
+
Push 'record' onto the input set so that jsawk will iterate over
|
|
178
|
+
it next.
|
|
179
|
+
|
|
180
|
+
params: Object|Array|Number|String record (the record to push)
|
|
181
|
+
return: void
|
|
182
|
+
|
|
183
|
+
json(thing)
|
|
184
|
+
Serialize 'thing' to JSON string.
|
|
185
|
+
|
|
186
|
+
params: Object|Array|Number|String thing (what to serialize)
|
|
187
|
+
return: String (the resulting JSON string)
|
|
188
|
+
|
|
189
|
+
uniq(array)
|
|
190
|
+
Return array of distinct elements.
|
|
191
|
+
|
|
192
|
+
params: Array array (the input array)
|
|
193
|
+
return: Array (the resulting array of distinct elements)
|
|
194
|
+
|
|
195
|
+
Q(query, thing)
|
|
196
|
+
Runs the JSONQuery 'query' on the JSON input 'thing'.
|
|
197
|
+
|
|
198
|
+
params: String query (the JSONQuery)
|
|
199
|
+
Array|Object thing (the JSON input)
|
|
200
|
+
return: Array|Object (result of running the query)
|
|
201
|
+
|
|
202
|
+
err(thing)
|
|
203
|
+
Print arguments (JSON encoded, if necessary) to stderr.
|
|
204
|
+
|
|
205
|
+
params: Object|Array|Number|String thing (what to encode)
|
|
206
|
+
return: void
|
|
207
|
+
|
|
208
|
+
out(thing)
|
|
209
|
+
Print arguments (JSON encoded, if necessary) to stdout.
|
|
210
|
+
|
|
211
|
+
params: Object|Array|Number|String thing (what to encode)
|
|
212
|
+
return: void
|
|
213
|
+
|
|
214
|
+
Errors and Output
|
|
215
|
+
-----------------
|
|
216
|
+
|
|
217
|
+
Errors in parsing scripts, JSON queries, or JSON input, and errors executing
|
|
218
|
+
scripts will all result in the appropriate error message on stderr, and
|
|
219
|
+
immediate exit with a non-zero exit status. Normal output is written to
|
|
220
|
+
stdout, unless the `-n` option is specified. In that case only output from
|
|
221
|
+
the `out()` or `err()` functions and error messages will appear.
|
|
222
|
+
|
|
223
|
+
Exit Status
|
|
224
|
+
-----------
|
|
225
|
+
|
|
226
|
+
On successful completion jsawk returns an exit status of `0`. If an error
|
|
227
|
+
ocurred and execution was aborted, a non-zero exit status will be returned.
|
|
228
|
+
|
|
229
|
+
### Exit Status
|
|
230
|
+
|
|
231
|
+
* **0** Successful completion.
|
|
232
|
+
* **1** Command line parsing error.
|
|
233
|
+
* **2** JSON parsing error.
|
|
234
|
+
* **3** Script error.
|
|
235
|
+
* **4** JSONQuery parsing error.
|
|
236
|
+
* **5** JSON stringify error.
|
|
237
|
+
|
|
238
|
+
JSONQuery
|
|
239
|
+
=========
|
|
240
|
+
|
|
241
|
+
Jsawk supports JSONQuery with the `-q` option. You can do almost anything
|
|
242
|
+
with JSONQuery that you can do with jsawk scripts, to include selecting
|
|
243
|
+
records, drilling down into records, mapping input sets to output sets as
|
|
244
|
+
a sort of filter, modifying the JSON, sorting, whathaveyou. JSONQuery is
|
|
245
|
+
to JSONPath is to JSON, as XQuery is to XPath is to XML. Here are some
|
|
246
|
+
JSONQuery resources to get started with this powerful tool:
|
|
247
|
+
|
|
248
|
+
* [The persevere JSONQuery documentation](http://docs.persvr.org/documentation/jsonquery)
|
|
249
|
+
* [Kris Zyp's intro to JSONQuery in dojo](http://www.sitepen.com/blog/2008/07/16/jsonquery-data-querying-beyond-jsonpath/)
|
|
250
|
+
|
|
251
|
+
Examples
|
|
252
|
+
========
|
|
253
|
+
|
|
254
|
+
For the following examples, suppose there is a file `/tmp/t`, with the
|
|
255
|
+
following contents:
|
|
256
|
+
|
|
257
|
+
[
|
|
258
|
+
{
|
|
259
|
+
"first" : "trevor",
|
|
260
|
+
"last" : "wellington",
|
|
261
|
+
"from" : "england",
|
|
262
|
+
"age" : 52,
|
|
263
|
+
"sports" : [ "rugby", "badmitton", "snooker" ]
|
|
264
|
+
},
|
|
265
|
+
{
|
|
266
|
+
"first" : "yoni",
|
|
267
|
+
"last" : "halevi",
|
|
268
|
+
"from" : "israel",
|
|
269
|
+
"age" : 26,
|
|
270
|
+
"sports" : [ "soccer", "windsurfing" ]
|
|
271
|
+
},
|
|
272
|
+
{
|
|
273
|
+
"first" : "cory",
|
|
274
|
+
"last" : "parker",
|
|
275
|
+
"from" : "united states",
|
|
276
|
+
"age" : 31,
|
|
277
|
+
"sports" : [ "windsurfing", "baseball", "extreeeeme kayaking" ]
|
|
278
|
+
}
|
|
279
|
+
]
|
|
280
|
+
|
|
281
|
+
This is going to be the input JSON text we will use in the examples.
|
|
282
|
+
|
|
283
|
+
JSON-to-JSON Transformations
|
|
284
|
+
----------------------------
|
|
285
|
+
|
|
286
|
+
These examples transform the input JSON, modifying it and returning the
|
|
287
|
+
modified JSON as output on stdout to be piped elsewhere. Transformations of
|
|
288
|
+
this type are generally done with a script that follows one of these simple
|
|
289
|
+
patterns:
|
|
290
|
+
|
|
291
|
+
1. Modify the `this` object in place (no `return` statement necessary).
|
|
292
|
+
1. Create a replacement object for each record, and `return` it at the end
|
|
293
|
+
of each iteration.
|
|
294
|
+
|
|
295
|
+
These patterns leave the records in JSON format, and they are automatically
|
|
296
|
+
printed to stdout without the use of the `out()` function.
|
|
297
|
+
|
|
298
|
+
### The Identity Mapping
|
|
299
|
+
|
|
300
|
+
This is the identity transformation: it doesn't really do anything other
|
|
301
|
+
than pass the input straight through.
|
|
302
|
+
|
|
303
|
+
cat /tmp/t | jsawk
|
|
304
|
+
|
|
305
|
+
You should get the input back out, unmolested.
|
|
306
|
+
|
|
307
|
+
### Increment Everyone's Age
|
|
308
|
+
|
|
309
|
+
Looks like it's everyone's birthday today. We'll take the JSON input and
|
|
310
|
+
increment each object's `age` property, sending the resulting JSON output to
|
|
311
|
+
stdout.
|
|
312
|
+
|
|
313
|
+
cat /tmp/t | jsawk 'this.age++'
|
|
314
|
+
|
|
315
|
+
Notice that there is no need to write `return this` in the script. That is
|
|
316
|
+
assumed---the runtime does it for you automatically if you don't explicitly
|
|
317
|
+
call `return` yourself.
|
|
318
|
+
|
|
319
|
+
### Flatten The "Sports" Array Of Each Element
|
|
320
|
+
|
|
321
|
+
Here we modify the input by replacing the `sports` property of each object
|
|
322
|
+
in the input array (the `sports` property is itself an array of strings) with
|
|
323
|
+
a single string containing all of the person's sports, separated by commas.
|
|
324
|
+
|
|
325
|
+
cat /tmp/t | jsawk 'this.sports = this.sports.join(",")'
|
|
326
|
+
|
|
327
|
+
Notice how altering the `this` object in place alters the result array
|
|
328
|
+
accordingly.
|
|
329
|
+
|
|
330
|
+
### Extract Only The "Age" Property Of Each Element ###
|
|
331
|
+
|
|
332
|
+
Normally we would modify the input set in place, by manipulating the `this`
|
|
333
|
+
object, which would be returned by default after each iteration. However,
|
|
334
|
+
sometimes we want only a single field from the input set.
|
|
335
|
+
|
|
336
|
+
cat /tmp/t | jsawk 'return this.age'
|
|
337
|
+
|
|
338
|
+
Putting a return statement in the script expression causes the default
|
|
339
|
+
return of `this` to be short-circuited, replacing this element with the
|
|
340
|
+
return value in the output set.
|
|
341
|
+
|
|
342
|
+
### JSON Grep: Select Certain Elements From Input ###
|
|
343
|
+
|
|
344
|
+
Sometimes you want to use awk to select certain records from the input set,
|
|
345
|
+
leaving the rest unchanged. This is like the `grep` pattern of operation. In
|
|
346
|
+
this example we will extract all the records corresponding to people who are
|
|
347
|
+
over 30 years old.
|
|
348
|
+
|
|
349
|
+
cat /tmp/t | jsawk 'if (this.age <= 30) return null'
|
|
350
|
+
|
|
351
|
+
This demonstrates how you can remove records from the results array by
|
|
352
|
+
returning a null value from your script.
|
|
353
|
+
|
|
354
|
+
Aggregate Functions
|
|
355
|
+
-------------------
|
|
356
|
+
|
|
357
|
+
Before and after scripts can be used to manipulate the JSON working set as
|
|
358
|
+
a whole, somewhat similar to the way aggregate functions like `SUM()` or
|
|
359
|
+
`COUNT()` work in SQL. These types of operations fall under a few basic
|
|
360
|
+
patterns.
|
|
361
|
+
|
|
362
|
+
1. Use a before script (`-b` option) to do things to the JSON input before
|
|
363
|
+
transformations are done by the main script.
|
|
364
|
+
2. Use an after script (`-a` option) to do things to the JSON result set
|
|
365
|
+
after all transformations are completed by the main script.
|
|
366
|
+
|
|
367
|
+
### Count How Many Elements Are In The Input Array
|
|
368
|
+
|
|
369
|
+
Here we use an after script to modify the result set, like this:
|
|
370
|
+
|
|
371
|
+
cat /tmp/t | jsawk -a 'return this.length'
|
|
372
|
+
|
|
373
|
+
Notice how the entire results array is replaced by the single number and
|
|
374
|
+
printed to stdout.
|
|
375
|
+
|
|
376
|
+
### Get a Sorted, Unique List of All Sports
|
|
377
|
+
|
|
378
|
+
This is an example of a JSON-to-JSON transformation that uses an after
|
|
379
|
+
script to manipulate the result set. It should produce an array of all
|
|
380
|
+
sports played by the people in the input set, sorted lexically, and with
|
|
381
|
+
all duplicate elements removed.
|
|
382
|
+
|
|
383
|
+
cat /tmp/t \
|
|
384
|
+
| jsawk 'RS=RS.concat(this.sports); return null' -a 'return uniq(RS).sort()'
|
|
385
|
+
|
|
386
|
+
Note the use of `return null` to prevent jsawk from adding the `this`
|
|
387
|
+
object to the result set automatically. Instead we manipulated the result
|
|
388
|
+
set explicitly, enabling each iteration to add more that one element to
|
|
389
|
+
it---the entire `sports` array. Also, notice the use of an after script
|
|
390
|
+
to sort the result set and remove duplicates.
|
|
391
|
+
|
|
392
|
+
JSON-to-Text Transformations
|
|
393
|
+
----------------------------
|
|
394
|
+
|
|
395
|
+
In the following examples we will be manipulating the JSON input to
|
|
396
|
+
produce text output instead of JSON, for cases where you will be extracting
|
|
397
|
+
information from a JSON data source and piping it to non JSON-accepting
|
|
398
|
+
processes elsewhere.
|
|
399
|
+
|
|
400
|
+
It is frequently useful to supress the regular JSON output when doing
|
|
401
|
+
JSON-to-Text transformations like these, with the `-n` option.
|
|
402
|
+
|
|
403
|
+
### Get A List Of All Sports
|
|
404
|
+
|
|
405
|
+
This one generates a list of all the sports that are played by the people
|
|
406
|
+
in our little JSON list, one per line, without duplicate entries, sorted
|
|
407
|
+
alphabetically.
|
|
408
|
+
|
|
409
|
+
cat /tmp/t \
|
|
410
|
+
| jsawk -a 'return this.join("\n")' 'return this.sports.join("\n")' \
|
|
411
|
+
| sort -u
|
|
412
|
+
|
|
413
|
+
Notice the use of JSONQuery to drill down into the JSON objects, an "after"
|
|
414
|
+
script to collate the results, and everything piped to the Unix `sort`
|
|
415
|
+
tool to remove duplicate entries and do the lexical ordering. This is
|
|
416
|
+
starting to show the power of the awk-like behavior now.
|
|
417
|
+
|
|
418
|
+
### Return a Boolean Value
|
|
419
|
+
|
|
420
|
+
Sometimes you want to just check for a certain condition in a shell script.
|
|
421
|
+
Suppose you want to know if there are any people over the age of 50 in the
|
|
422
|
+
JSON input array, like this:
|
|
423
|
+
|
|
424
|
+
jsawk -n 'if (this.age > 50) quit(1)' < /tmp/t || echo "We have people over 50 here---naptime in effect."
|
|
425
|
+
|
|
426
|
+
We suppress normal result set output with `-n` and use the `quit()` function
|
|
427
|
+
to return a value in the exit status. The default exit status is, of course,
|
|
428
|
+
zero for success.
|
|
429
|
+
|
|
430
|
+
JSON Pretty-Printing
|
|
431
|
+
====================
|
|
432
|
+
|
|
433
|
+
[Resty](http://github.com/micha/resty) includes the `pp` script that will
|
|
434
|
+
pretty-print JSON for you. You just need to install the JSON perl module
|
|
435
|
+
from CPAN. Use it like this:
|
|
436
|
+
|
|
437
|
+
GET /blogs.json | jsawk -q '..author' | pp
|