jeg 0.2.1
Sign up to get free protection for your applications and to get access to all the features.
- data/LICENSE +10 -0
- data/README.mkd +80 -0
- data/Rakefile +26 -0
- data/VERSION +1 -0
- data/bin/jeg +158 -0
- data/jeg.gemspec +53 -0
- data/pkg/jeg-0.2.1.gem +0 -0
- data/test/docs.rb +26 -0
- metadata +72 -0
data/LICENSE
ADDED
@@ -0,0 +1,10 @@
|
|
1
|
+
THE DRINK-WARE LICENSE (forked from BEER-WARE r42)
|
2
|
+
|
3
|
+
Gabriele Renzi (http://www.riffraff.info)
|
4
|
+
wrote this code, whereas not specified otherwise.
|
5
|
+
|
6
|
+
As long as you retain this notice you can do whatever you want with this stuff.
|
7
|
+
If we meet some day, and you think this stuff is worth it, you can buy me
|
8
|
+
a beer, coffee, palinka shot or any other drink in return.
|
9
|
+
|
10
|
+
|
data/README.mkd
ADDED
@@ -0,0 +1,80 @@
|
|
1
|
+
jeg, a json grep
|
2
|
+
===========
|
3
|
+
|
4
|
+
Jeg was born from the repeated annoyance of trying to quickly glance at
|
5
|
+
some JSON-fomatted data and being unable to really get it, then have to
|
6
|
+
pass it through a javascript formatter and then later go and trying to
|
7
|
+
find the data that I alread know was under the "name" field but was
|
8
|
+
impossible to discern in a 800 character single line string.
|
9
|
+
|
10
|
+
Ah, how I craved for the simplicty of tabular data that I could slice
|
11
|
+
and dice with grep and cut.
|
12
|
+
|
13
|
+
Thus, Jeg was born.
|
14
|
+
|
15
|
+
So what is it, concretely?
|
16
|
+
--------------------------
|
17
|
+
|
18
|
+
Simply put, it allows you to select parts of a json structure using
|
19
|
+
JSONPath. For example, if you want to find a user's real name on twitter from
|
20
|
+
this
|
21
|
+
|
22
|
+
{"url":"http://www.riffraff.info","description":"Code Monkey, Professional Student, Geek","time_zone":"Rome","profile_sidebar_fill_color":"e0ff92","status":{"in_reply_to_user_id":null,"in_reply_to_status_id":null,"in_reply_to_screen_name":null,"created_at":"Sun Jan 24 16:16:28 +0000 2010","source":"<a href=\"http://www.tweetdeck.com/\" rel=\"nofollow\">TweetDeck</a>","truncated":false,"id":8153979023,"favorited":false,"text":"darn I have to publish a forked gem, and can't understand the state of the art in how to do so :("},"statuses_count":447,"created_at":"Tue Jan 02 11:27:37 +0000 2007","profile_sidebar_border_color":"87bc44","contributors_enabled":false,"favourites_count":3,"followers_count":142,"profile_image_url":"http://a3.twimg.com/profile_images/88180929/nyussi_normal.jpg","profile_text_color":"000000","lang":"en","geo_enabled":true,"notifications":null,"profile_background_image_url":"http://s.twimg.com/a/1264119427/images/themes/theme1/bg.png","friends_count":171,"protected":false,"screen_name":"riffraff","following":null,"profile_link_color":"0000ff","location":"milan/rome/budapest","name":"gabriele renzi","verified":false,"profile_background_tile":false,"id":446303,"utc_offset":3600,"profile_background_color":"9ae4e8"
|
23
|
+
|
24
|
+
|
25
|
+
you can simply query the api via curl and use jeg to extract the name
|
26
|
+
field:
|
27
|
+
|
28
|
+
$ curl http://twitter.com/users/show.json?screen_name=riffraff -s | jeg name
|
29
|
+
gabriele renzi
|
30
|
+
|
31
|
+
Or if you want the 'text' fields in a list of objects from the Cascaad
|
32
|
+
APIs
|
33
|
+
|
34
|
+
$ curl api.cascaad.com/2/messages/latest.json?type=NEWS -s | jeg $..text
|
35
|
+
counter the Tea Party movement, which is abt stopping things, with Innovation Movement, which is abt starting things. http://bit.ly/4o35m4
|
36
|
+
Bringing Silicon Valley to Sacramento: Why Entrepreneurs Need to Help Rebuild California's IT Systems http://tcrn.ch/8pD3Ki by @vwadhwa
|
37
|
+
Spectacular Thomas Friedman OpEd on making 2010 the Year of Innovation and Start-Up America - http://nyti.ms/4qk8Jh
|
38
|
+
|
39
|
+
|
40
|
+
|
41
|
+
Just this?
|
42
|
+
----------
|
43
|
+
|
44
|
+
Well, a bit more, see `jeg -h`
|
45
|
+
|
46
|
+
|
47
|
+
Installation
|
48
|
+
------------
|
49
|
+
|
50
|
+
Jeg depends on the riffraff_jsonpath and json libraries.
|
51
|
+
The former is a small fork of the original jsonpath gem without warnings and with a couple of tiny fixes.
|
52
|
+
|
53
|
+
You can just dump the single `jeg` script in some directory in your path and make it executable to install it,
|
54
|
+
or you can user rubygems
|
55
|
+
|
56
|
+
gem install jeg
|
57
|
+
|
58
|
+
And that's it.
|
59
|
+
|
60
|
+
Running the tests
|
61
|
+
-----------------
|
62
|
+
As of now no unit tests were written but the command line examples are extracted and executed by
|
63
|
+
the code in `test/` (if you have a checkout and you are reading this file) just run them with
|
64
|
+
ruby having the script in your path, or use `rake test` you shall see something like
|
65
|
+
|
66
|
+
Loaded suite test/docs
|
67
|
+
Started
|
68
|
+
...............
|
69
|
+
Finished in 9.999814 seconds.
|
70
|
+
|
71
|
+
15 tests, 15 assertions, 0 failures, 0 errors
|
72
|
+
|
73
|
+
If you don't please report it at http://github.com/riffraff/jeg/issues
|
74
|
+
|
75
|
+
License
|
76
|
+
-------
|
77
|
+
|
78
|
+
jeg is free software (pretend you wrote it, sell it for huge profit) but see
|
79
|
+
accompanying LICENSE file for the full text of the license.
|
80
|
+
|
data/Rakefile
ADDED
@@ -0,0 +1,26 @@
|
|
1
|
+
require 'rubygems'
|
2
|
+
require 'rake'
|
3
|
+
begin
|
4
|
+
require 'jeweler'
|
5
|
+
Jeweler::Tasks.new do |s|
|
6
|
+
s.name = "jeg"
|
7
|
+
s.executables = "jeg"
|
8
|
+
s.summary = "jeg a json grep"
|
9
|
+
s.email = "rff.rff@gmail.com"
|
10
|
+
s.homepage = "http://github.com/riffraff/jeg"
|
11
|
+
s.description = "A command line tool to slice and dice JSON-encoded data as simple a structured text format"
|
12
|
+
s.authors = ["Gabriele Renzi"]
|
13
|
+
s.files = FileList["**/*"]
|
14
|
+
s.add_dependency 'riffraff_jsonpath'
|
15
|
+
end
|
16
|
+
rescue LoadError
|
17
|
+
puts "Jeweler, or one of its dependencies, is not available. Install it with: sudo gem install technicalpickles-jeweler -s http://gems.github.com"
|
18
|
+
end
|
19
|
+
|
20
|
+
require 'rake/testtask'
|
21
|
+
Rake::TestTask.new do |t|
|
22
|
+
t.test_files = FileList['test/*.rb']
|
23
|
+
t.verbose = true
|
24
|
+
end
|
25
|
+
|
26
|
+
|
data/VERSION
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
0.2.1
|
data/bin/jeg
ADDED
@@ -0,0 +1,158 @@
|
|
1
|
+
#!/usr/bin/env ruby -w
|
2
|
+
# vim: syntax=ruby ts=2 sts=2 expandtab
|
3
|
+
#
|
4
|
+
|
5
|
+
require 'rubygems'
|
6
|
+
require 'json'
|
7
|
+
require 'jsonpath'
|
8
|
+
require 'optparse'
|
9
|
+
|
10
|
+
class Jeg
|
11
|
+
Version = 0.2
|
12
|
+
|
13
|
+
attr :options
|
14
|
+
attr :input
|
15
|
+
|
16
|
+
def mk_expr(v)
|
17
|
+
v = '.'+v if v =~ /^\w/
|
18
|
+
v = "$"+v if v =~ /^\./
|
19
|
+
v
|
20
|
+
end
|
21
|
+
|
22
|
+
def initialize(arguments=ARGV, input=ARGF)
|
23
|
+
@input = input
|
24
|
+
@options = {:format=>:smart}
|
25
|
+
OptionParser.new(arguments) do |opts|
|
26
|
+
opts.banner = <<-Banner
|
27
|
+
Usage: jeg [options] [file]"
|
28
|
+
|
29
|
+
jeg interprets JSONPath queries on a JSON stream (line-separated blocks).
|
30
|
+
It is designed for quick & dirty work so it assumes things for you.
|
31
|
+
The default output (--smart) will print simple strings if the result is a single basic type, one per line
|
32
|
+
(grep style) while if the match is structured data it will be presented as json.
|
33
|
+
Also, by default if the JSONPath expression does not start with $. (top element) it will be added automatically.
|
34
|
+
You can override this with a switch.
|
35
|
+
|
36
|
+
Examples:
|
37
|
+
Select a top level field in top struct
|
38
|
+
{"text":"babble babble"} | jeg -e $.text => babble babble
|
39
|
+
{"text":"babble babble"} | jeg -e $.text -f json => ["babble babble"]
|
40
|
+
{"text":"babble babble"} | jeg -e $.text -f ruby => ["babble babble"]
|
41
|
+
{"obj":{"key":"value","index":1}} | jeg -e $.obj -f smart => {"index":1,"key":"value"}
|
42
|
+
{"obj":{"key":"value","index":1}} | jeg -e $.obj -f json => [{"key":"value","index":1}]
|
43
|
+
{"obj":{"key":"value","index":1}} | jeg -e $.obj -f ruby => [{"key"=>"value","index"=>1}]
|
44
|
+
The -e option is implicit if none is provided
|
45
|
+
{"text":"babble babble"} | jeg $.text => babble babble
|
46
|
+
|
47
|
+
Leading '$' ("top") and '.' ("this object") can be omitted
|
48
|
+
{"text":"babble babble"} | jeg -e .text => babble babble
|
49
|
+
{"text":"babble babble"} | jeg -e text => babble babble
|
50
|
+
Select subfield
|
51
|
+
{"results":[{"text":"foo"},{"text":"bar"}]} | jeg results.text -f json => []
|
52
|
+
{"results":[{"text":"foo"},{"text":"bar"}]} | jeg results..text -f json => ["foo","bar"]
|
53
|
+
Array access
|
54
|
+
{"results":[{"a":"b"},{"a":"d"}]} | jeg results[0] -f json => [{"a":"b"}]
|
55
|
+
|
56
|
+
Select a sub field everywhere, most always you want this
|
57
|
+
{"result":[{"text":"babble babble"}]} | jeg -e ..text => babble babble
|
58
|
+
So you also have a shortcut
|
59
|
+
{"result":[{"text":"babble babble"}]} | jeg -a text => babble babble
|
60
|
+
|
61
|
+
Select all equal fields in an array element
|
62
|
+
{"ary":[[{"name":"joe"}, {"name":"jean"}, {"name":"jane"}],[{"name":"wally"}]]} | jeg -e ary[0]..name -f json => ["joe", "jean", "jane"]
|
63
|
+
|
64
|
+
|
65
|
+
See JSONPath specs at http://goessner.net/articles/JsonPath/ .
|
66
|
+
|
67
|
+
|
68
|
+
Banner
|
69
|
+
|
70
|
+
opts.on("-v", "--[no-]verbose", "Run verbosely") do |v|
|
71
|
+
options[:verbose] = v
|
72
|
+
end
|
73
|
+
opts.on("-a", "--anywhere FIELD", "Matches a field anywhere") do |v|
|
74
|
+
options[:expression] = "$.." + v
|
75
|
+
end
|
76
|
+
opts.on("-e", "--expression JSONPATHEXPR", <<-Eof.strip ) do |v|
|
77
|
+
Specify a jsonpath expr, leading '$' or '.' are automatically added
|
78
|
+
Eof
|
79
|
+
options[:expression] = mk_expr(v)
|
80
|
+
end
|
81
|
+
opts.on("-x", "--explain", "Show what jeg is doing to your input and what is the actual expression used") do |v|
|
82
|
+
options[:explain] = v
|
83
|
+
end
|
84
|
+
opts.on("-p", "--pretty-print", "Print the input objects as pretty ruby") do |v|
|
85
|
+
options[:pretty_print] = v
|
86
|
+
end
|
87
|
+
opts.on_tail("-f", "--format FORMAT", [:smart, :json, :ruby], "Select output format (smart, json ruby)" ) do |v|
|
88
|
+
options[:format] = v
|
89
|
+
end
|
90
|
+
opts.on_tail("-h", "--help", "Show this message") do
|
91
|
+
puts opts
|
92
|
+
exit
|
93
|
+
end
|
94
|
+
|
95
|
+
# Another typical switch to print the version.
|
96
|
+
opts.on_tail("--version", "Show version") do
|
97
|
+
puts Version
|
98
|
+
exit
|
99
|
+
end
|
100
|
+
end.parse!
|
101
|
+
|
102
|
+
def print_smart(list)
|
103
|
+
list.each do |arg|
|
104
|
+
if arg.is_a?(Array) || arg.is_a?(Hash)
|
105
|
+
puts arg.to_json
|
106
|
+
else
|
107
|
+
puts arg
|
108
|
+
end
|
109
|
+
end
|
110
|
+
end
|
111
|
+
|
112
|
+
def loop
|
113
|
+
|
114
|
+
if options[:expression].nil?
|
115
|
+
options[:expression] = mk_expr(ARGV.shift)
|
116
|
+
end
|
117
|
+
if options[:pretty_print]
|
118
|
+
require 'pp'
|
119
|
+
end
|
120
|
+
if options[:explain]
|
121
|
+
puts "Using path '#{options[:expression]}'"
|
122
|
+
end
|
123
|
+
input.each do |line|
|
124
|
+
begin
|
125
|
+
json = JSON.parse(line)
|
126
|
+
if options[:verbose]
|
127
|
+
puts line
|
128
|
+
puts json
|
129
|
+
end
|
130
|
+
if options[:pretty_print]
|
131
|
+
pp json
|
132
|
+
end
|
133
|
+
res = JSONPath.lookup(json, options[:expression])
|
134
|
+
case options[:format]
|
135
|
+
when :smart
|
136
|
+
print_smart(res)
|
137
|
+
when :json
|
138
|
+
puts res.to_json
|
139
|
+
when :ruby
|
140
|
+
p res
|
141
|
+
else
|
142
|
+
abort("invalid format")
|
143
|
+
end
|
144
|
+
|
145
|
+
rescue JSON::ParserError=>e
|
146
|
+
STDERR.puts "Something went wrong parsing:\n#{line}"
|
147
|
+
STDERR.puts e.message
|
148
|
+
end
|
149
|
+
end
|
150
|
+
end
|
151
|
+
end
|
152
|
+
end
|
153
|
+
|
154
|
+
if __FILE__==$0
|
155
|
+
# To get rid of the annoying stack trace on ctrl-C:
|
156
|
+
trap("INT") { abort }
|
157
|
+
Jeg.new.loop
|
158
|
+
end
|
data/jeg.gemspec
ADDED
@@ -0,0 +1,53 @@
|
|
1
|
+
# Generated by jeweler
|
2
|
+
# DO NOT EDIT THIS FILE DIRECTLY
|
3
|
+
# Instead, edit Jeweler::Tasks in Rakefile, and run the gemspec command
|
4
|
+
# -*- encoding: utf-8 -*-
|
5
|
+
|
6
|
+
Gem::Specification.new do |s|
|
7
|
+
s.name = %q{jeg}
|
8
|
+
s.version = "0.2.1"
|
9
|
+
|
10
|
+
s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
|
11
|
+
s.authors = ["Gabriele Renzi"]
|
12
|
+
s.date = %q{2010-02-03}
|
13
|
+
s.default_executable = %q{jeg}
|
14
|
+
s.description = %q{A command line tool to slice and dice JSON-encoded data as simple a structured text format}
|
15
|
+
s.email = %q{rff.rff@gmail.com}
|
16
|
+
s.executables = ["jeg"]
|
17
|
+
s.extra_rdoc_files = [
|
18
|
+
"LICENSE",
|
19
|
+
"README.mkd"
|
20
|
+
]
|
21
|
+
s.files = [
|
22
|
+
"LICENSE",
|
23
|
+
"README.mkd",
|
24
|
+
"Rakefile",
|
25
|
+
"VERSION",
|
26
|
+
"bin/jeg",
|
27
|
+
"jeg.gemspec",
|
28
|
+
"pkg/jeg-0.2.1.gem",
|
29
|
+
"test/docs.rb"
|
30
|
+
]
|
31
|
+
s.homepage = %q{http://github.com/riffraff/jeg}
|
32
|
+
s.rdoc_options = ["--charset=UTF-8"]
|
33
|
+
s.require_paths = ["lib"]
|
34
|
+
s.rubygems_version = %q{1.3.5}
|
35
|
+
s.summary = %q{jeg a json grep}
|
36
|
+
s.test_files = [
|
37
|
+
"test/docs.rb"
|
38
|
+
]
|
39
|
+
|
40
|
+
if s.respond_to? :specification_version then
|
41
|
+
current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION
|
42
|
+
s.specification_version = 3
|
43
|
+
|
44
|
+
if Gem::Version.new(Gem::RubyGemsVersion) >= Gem::Version.new('1.2.0') then
|
45
|
+
s.add_runtime_dependency(%q<riffraff_jsonpath>, [">= 0"])
|
46
|
+
else
|
47
|
+
s.add_dependency(%q<riffraff_jsonpath>, [">= 0"])
|
48
|
+
end
|
49
|
+
else
|
50
|
+
s.add_dependency(%q<riffraff_jsonpath>, [">= 0"])
|
51
|
+
end
|
52
|
+
end
|
53
|
+
|
data/pkg/jeg-0.2.1.gem
ADDED
Binary file
|
data/test/docs.rb
ADDED
@@ -0,0 +1,26 @@
|
|
1
|
+
require 'test/unit'
|
2
|
+
require 'rubygems'
|
3
|
+
require 'json'
|
4
|
+
abort("jeg is not in path") if %x{jeg -h}.empty?
|
5
|
+
|
6
|
+
class T < Test::Unit::TestCase
|
7
|
+
jeg_file=%x(which jeg).chomp
|
8
|
+
examples = File.readlines(jeg_file).grep(/jeg .*=>/)
|
9
|
+
examples.each_with_index do |line,idx|
|
10
|
+
define_method("test_#{idx}") do
|
11
|
+
#p line
|
12
|
+
cmd, exp = line.scan(%r-(.*?)=>(.*)-)[0]
|
13
|
+
json, pipe = cmd.split('|')
|
14
|
+
res = %x{echo '#{json}' | #{pipe}}
|
15
|
+
#print 'json:'+json, 'pipe:'+pipe, 'exp:'+exp, 'res:'+res, "\n"
|
16
|
+
if pipe =~ /-f json/
|
17
|
+
assert_equal JSON.parse(exp.strip), JSON.parse(res.strip),"#{json} failed with cmd #{cmd}"
|
18
|
+
elsif pipe =~ /-f ruby/
|
19
|
+
assert_equal eval(exp.strip), eval(res.strip),"#{json} failed with cmd #{cmd}"
|
20
|
+
else
|
21
|
+
assert_equal exp.strip, res.strip,"#{json} failed with cmd #{cmd}"
|
22
|
+
end
|
23
|
+
end
|
24
|
+
end
|
25
|
+
end
|
26
|
+
|
metadata
ADDED
@@ -0,0 +1,72 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: jeg
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.2.1
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Gabriele Renzi
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
|
12
|
+
date: 2010-02-03 00:00:00 +01:00
|
13
|
+
default_executable: jeg
|
14
|
+
dependencies:
|
15
|
+
- !ruby/object:Gem::Dependency
|
16
|
+
name: riffraff_jsonpath
|
17
|
+
type: :runtime
|
18
|
+
version_requirement:
|
19
|
+
version_requirements: !ruby/object:Gem::Requirement
|
20
|
+
requirements:
|
21
|
+
- - ">="
|
22
|
+
- !ruby/object:Gem::Version
|
23
|
+
version: "0"
|
24
|
+
version:
|
25
|
+
description: A command line tool to slice and dice JSON-encoded data as simple a structured text format
|
26
|
+
email: rff.rff@gmail.com
|
27
|
+
executables:
|
28
|
+
- jeg
|
29
|
+
extensions: []
|
30
|
+
|
31
|
+
extra_rdoc_files:
|
32
|
+
- LICENSE
|
33
|
+
- README.mkd
|
34
|
+
files:
|
35
|
+
- LICENSE
|
36
|
+
- README.mkd
|
37
|
+
- Rakefile
|
38
|
+
- VERSION
|
39
|
+
- bin/jeg
|
40
|
+
- jeg.gemspec
|
41
|
+
- pkg/jeg-0.2.1.gem
|
42
|
+
- test/docs.rb
|
43
|
+
has_rdoc: true
|
44
|
+
homepage: http://github.com/riffraff/jeg
|
45
|
+
licenses: []
|
46
|
+
|
47
|
+
post_install_message:
|
48
|
+
rdoc_options:
|
49
|
+
- --charset=UTF-8
|
50
|
+
require_paths:
|
51
|
+
- lib
|
52
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
53
|
+
requirements:
|
54
|
+
- - ">="
|
55
|
+
- !ruby/object:Gem::Version
|
56
|
+
version: "0"
|
57
|
+
version:
|
58
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
59
|
+
requirements:
|
60
|
+
- - ">="
|
61
|
+
- !ruby/object:Gem::Version
|
62
|
+
version: "0"
|
63
|
+
version:
|
64
|
+
requirements: []
|
65
|
+
|
66
|
+
rubyforge_project:
|
67
|
+
rubygems_version: 1.3.5
|
68
|
+
signing_key:
|
69
|
+
specification_version: 3
|
70
|
+
summary: jeg a json grep
|
71
|
+
test_files:
|
72
|
+
- test/docs.rb
|