daru 0.1.5 → 0.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: b584977ade16d5cfbd3bdd9da7202bbc2adae894
-  data.tar.gz: 3de8ef80fc55c91c0f504543fda1f23b40b4db6b
+SHA256:
+  metadata.gz: 264a0549062a2c6b062f8c031b4e03524fd1bd852d59927a54722e8b8e68a2e8
+  data.tar.gz: 2dee6ded3fb009045a6ef13203c8ebe4251458e64b3e41a737b8bc04d4d0b91f
 SHA512:
-  metadata.gz: 3f498f5caa4dbc14c2ca884c45492e474442d3f067ecf673a0235f01ec1781bb96c6f6e764006dcbf349a68ea011ccd977d69fdeb324a202ffa4fc5351a3a7b9
-  data.tar.gz: d209e60b566e0e900d9c62fca6a1fb9023cd09cd81194e9b331c59682f557d8917ca347978bd4ed9b614c8d1ba441126211f3dc15a691752733f5dca0200be5d
+  metadata.gz: 1cb5cf9a2aa1660e9cd0d0a286af6d3dcbe987b5001f15a90e5b89394c36fdaedd8e29e742f591c36de64e61363ba66ceff8ae9f075c2ce1a4352a56584b4c24
+  data.tar.gz: 8e3d5843f871c0fba685430e27ff91090db4873da8edbfe2d1095b12a24a06dcda7ae1b198b9a3f7fcc013aeb757560088bcbb507a70ece4a4976eda63094cd8

data/.github/ISSUE_TEMPLATE.md

@@ -0,0 +1,18 @@
+Heya! We are glad you are going to contribute to Daru by creating an issue, and kindly ask you to
+follow the simple rules:
+
+1. If it is a bug report, please provide a **self-containing** Ruby code for reproducing the bug.
+  This means if Daru contributors just copy-paste the code from issue into `this-is-bug.rb` and run
+  `ruby this-is-bug.rb`, it will be reproduced. If the bug is hard to spot (e.g. it is not some
+  `NoMethodError`, but the differences in data structure), please show it with comment in code or
+  plain text in the issue.
+2. If it is a feature request, try to do the following (if possible):
+  * show how new feature will work with small code example;
+  * explain the use case (if it is not 200% obvious);
+  * if you are aware of it, show how it works in pandas and/or R.
+3. If it is just a question ("how to do this or that" or "why Daru does this or that") feel free to
+  write it in any form that is convenient to you, but remember code examples and use cases are always
+  welcome.
+
+Thanks! And please remove this text when finished with your issue description :)
+

data/.gitignore

@@ -6,3 +6,4 @@ doc/
 vendor/
 profile/out/
 coverage/
+.ruby-version

data/.rubocop.yml

@@ -2,17 +2,22 @@ AllCops:
   Include:
     - 'lib/**/*'
   Exclude:
+    - 'daru.gemspec'
+    - 'Rakefile'
+    - 'Gemfile'
+    - 'Guardfile'
     - '**/*.erb'
     - 'spec/*'
     - 'spec/**/*'
     - 'vendor/**/*'
     - 'benchmarks/*'
     - 'profile/*'
+    - 'tmp/*'
   DisplayCopNames: true
   TargetRubyVersion: 2.0
 
 # Preferred codebase style ---------------------------------------------
-Style/ExtraSpacing:
+Layout/ExtraSpacing:
   AllowForAlignment: true
 
 Style/FormatString:
@@ -21,16 +26,16 @@ Style/FormatString:
 Style/AndOr:
   EnforcedStyle: conditionals
 
-Style/SpaceAroundEqualsInParameterDefault:
+Layout/SpaceAroundEqualsInParameterDefault:
   EnforcedStyle: no_space
 
-Style/SpaceInsideBlockBraces:
+Layout/SpaceInsideBlockBraces:
   EnforcedStyle: space
 
-Style/SpaceInsideHashLiteralBraces:
+Layout/SpaceInsideHashLiteralBraces:
   EnforcedStyle: no_space
 
-Style/AlignParameters:
+Layout/AlignParameters:
   EnforcedStyle: with_fixed_indentation
 
 Style/EmptyElse:
@@ -57,10 +62,10 @@ Style/SingleLineBlockParams:
 Style/PerlBackrefs:
   Enabled: false
 
-Style/SpaceAfterComma:
+Layout/SpaceAfterComma:
   Enabled: false
 
-Style/SpaceAroundOperators:
+Layout/SpaceAroundOperators:
   Enabled: false
 
 Style/EmptyCaseCondition:
@@ -69,6 +74,15 @@ Style/EmptyCaseCondition:
 Style/MultilineBlockChain:
   Enabled: false
 
+# See https://github.com/bbatsov/rubocop/issues/4429
+Style/YodaCondition:
+  Enabled: false
+
+Style/PercentLiteralDelimiters:
+  PreferredDelimiters:
+    '%i': '[]'
+    '%w': '[]'
+
 # Neither of prefered styles are good enough :(
 Style/BlockDelimiters:
   Enabled: false

data/.travis.yml

@@ -2,19 +2,24 @@ language:
   ruby
 
 rvm:
-  - '2.0'
-  - '2.1'
-  - '2.2'
-  - '2.3.0'
-  - '2.4.0'
+  - '2.5.1'
+  - '2.7.1'
 
 matrix:
+  allow_failures:
+    - rvm: '2.0'
   fast_finish:
     true
 
+env:
+  - DARU_TEST_NMATRIX=1 DARU_TEST_GSL=1
+
 script:
+  - bundle add yard-junk
+  - bundle install
   - bundle exec rspec
   - bundle exec rubocop
+  - bundle exec yard-junk
 
 install:
   - gem install bundler

data/CONTRIBUTING.md

@@ -6,15 +6,18 @@ Either nmatrix or rb-gsl are NOT NECESSARY for using daru. They are just require
 
 To install dependencies, execute the following commands:
 
-*  `sudo apt-get update -qq`
-*  `sudo apt-get install -y libgsl0-dev r-base r-base-dev`
-*  `sudo Rscript -e "install.packages(c('Rserve','irr'),,'http://cran.us.r-project.org')"`
-*  `sudo apt-get install libmagickwand-dev imagemagick`
-
-
-Then install remaining dependencies:
-
-  `bundle install`
+``` bash
+sudo apt-get update -qq
+sudo apt-get install -y libgsl0-dev r-base r-base-dev
+sudo Rscript -e "install.packages(c('Rserve','irr'),,'http://cran.us.r-project.org')"
+sudo apt-get install libmagickwand-dev imagemagick
+export DARU_TEST_NMATRIX=1  # for running nmatrix tests.
+export DARU_TEST_GSL=1 # for running rb-GSL tests.
+bundle install
+```
+You don't need `DARU_TEST_NMATRIX` or `DARU_TEST_GSL` if you don't want to make changes
+to those parts of the code. However, they will be set in CI and will raise a test failure
+if something goes wrong.
 
 And run the test suite (should be all green with pending tests):
 
@@ -22,16 +25,18 @@ And run the test suite (should be all green with pending tests):
 
 If you have problems installing nmatrix, please consult the [nmatrix installation wiki](https://github.com/SciRuby/nmatrix/wiki/Installation) or the [mailing list](https://groups.google.com/forum/#!forum/sciruby-dev).
 
+
 While preparing your pull requests, don't forget to check your code with Rubocop:
 
   `bundle exec rubocop`
   
 [Optional] Install all Ruby versions which Daru currently supports with `rake spec setup`.
 
+
 ## Basic Development Flow
 
 1. Create a new branch with `git checkout -b <branch_name>`.
-2. Make your changes.
+2. Make your changes. Write tests covering every case how your feature will be used. If creating new files for tests, refer to the 'Testing' section [below](#Testing).
 3. Try out these changes with `rake pry`.
 4. Run the test suite with `rake spec`. (Alternatively you can use `guard` as described [here](https://github.com/SciRuby/daru/blob/master/CONTRIBUTING.md#testing). Also run Rubocop coding style guidelines with `rake cop`.
 5. Commit the changes with `git commit -am "briefly describe what you did"` and submit pull request.

data/History.md

@@ -1,3 +1,125 @@
+# 0.3 (30 May 2020)
+* Major Enhacements
+  - Remove official support for Ruby < 2.5.1. Now we only test with 2.5.1 and 2.7.1. (@v0dro)
+  - Make nmatrix and gsl optional dependencies for testing. (@v0dro)
+  - Update sqlite, activerecord, nokogiri, packable, rake dependencies. (@v0dro)
+  - Remove runtime dependency on backports. (@v0dro)
+  - Add `Daru::Vector#match and Daru::Vector#apply_where` methods (@athityakumar).
+  - Add support for options to the `Daru` module. Adds a separate module `Daru::Configuration` that
+  can hold data for overall configuration of daru's execution. (@kojix2)
+* Minor Enhancements
+  - Add new `DataFrame#insert_vector` method. (@cyrillefr)
+  - Add `Vector#last`. (@kojix2)
+  - Add `DataFrame#rename_vectors!`. (@neumanrq)
+  - Refactor `GroupBy#apply_method`. (@paisible-wanderer)
+  - Auto-adjust header parameters when printing to terminal. (@ncs1)
+  - Infer offsets of timeseries automatically when they are a natural number multiple of seconds. (@jpaulgs)
+
+# 0.2.2 (8 August 2019)
+
+* Minor Enhancements
+  - DataFrame#set_index can take column name array, which results in multi-index  https://github.com/SciRuby/daru/pull/471 (by @Yuki-Inoue)
+  - implements DataFrame#reset_index https://github.com/SciRuby/daru/pull/473  (by @Yuki-Inoue)
+  - Make DataFrame.from_activerecord faster https://github.com/SciRuby/daru/pull/464 (by @paisible-wanderer )
+  - Added access_row_tuples_by_indexs method https://github.com/SciRuby/daru/pull/463 (by @Prakriti-nith )
+
+* Fixes
+  - Fix reindex vector on argument error https://github.com/SciRuby/daru/pull/470 (by @Yuki-Inoue)
+  - Optimize aggregation https://github.com/SciRuby/daru/pull/464 (by @paisible-wanderer)
+  - Index#dup should copy reference to name too https://github.com/SciRuby/daru/pull/477 (by @Yuki-Inoue)
+  - Should support bundler version 2.x.x https://github.com/SciRuby/daru/pull/483/ (by @Shekharrajak )
+  - fix table style  https://github.com/SciRuby/daru/pull/489 (by @kojix2 )
+
+# 0.2.1 (02 July 2018)
+
+* Minor Enhancements
+  - Allow pasing singular Symbol to CSV converters option (@takkanm)
+  - Support calling GroupBy#each_group w/o blocks (@hibariya)
+  - Refactor grouping and aggregation (@paisible-wanderer)
+  - Add String Converter to Daru::IO::CSV::CONVERTERS (@takkanm)
+  - Fix annoying missing libraries warning
+  - Remove post-install message (nice yet useless)
+
+* Fixes
+  - Fix group_by for DataFrame with single row (@baarkerlounger)
+  - `#rolling_fillna!` bugfixes on `Daru::Vector` and `Daru::DataFrame` (@mhammiche)
+  - Fixes `#include?` on multiindex (@rohitner)
+
+# 0.2.0 (31 October 2017)
+* Major Enhancements
+  - Add `DataFrame#which` query DSL (experimental! @rainchen)
+  - Add `DataFrame/Vector#rolling_fillna` (@baarkerlounger)
+  - Add `GroupBy#aggregate` (@shekharrajak)
+  - Add `DataFrame#uniq` (@baarkerlounger)
+
+* Minor Enhancements
+  - Allow `Vector#count` to be called without param for category type Vector (@rainchen)
+  - Add option to `DataFrame#vector_sum` to skip nils (@parthm)
+  - Add installation instructions to README.md (@koishimasato)
+  - Add release policy documentation (@baarkerlounger)
+  - Set index as DataFrame's default x axis for nyaplot (@matugm)
+
+* Fixes
+  - Fix `DataFrame/Vector#to_s` when name is a symbol (@baarkerlounger)
+  - Force `Vector#proportions` to return float (@rainchen)
+  - `DataFrame#new` creates empty DataFrame when given empty hash (@parthm)
+  - Remove unnecessary backports dependencies (@zverok)
+  - Specify minimum packable dependency (@zverok)
+  - Preserve key/column order when creating DataFrame from hash (@baarkerlounger)
+  - Fix `DataFrame#add_row` for DF with multi-index (@zverok)
+  - Fix `Vector#min, `#max`, `#index_of_min`, `#index_of_max` (0.1.6 regression) (@athityakumar)
+  - Integrate yard-junk into CI (@rohitner)
+  - Remove Travis spec restriction (@zverok)
+  - Fix tuple sorting for DataFrames with nils (@baarkerlounger)
+  - Fix merge on index dropping default index (@rohitner)
+
+# 0.1.6 (04 August 2017)
+* Major Enhancements
+  - Add support for reading HTML tables into DataFrames (@athityakumar)
+  - Add support for importing remote CSVs (@athityakumar, @anshuman23)
+  - Allow named indexes (@Shekharrajak)
+  - DataFrame GroupBy returns MultiIndex DataFrame (@Shekharrajak)
+  - Add new functions to Vector: max, min, index_of_max, index_of_min, max_by, min_by, index_of_max_by, index_of_min_by (@athityakumar)
+  - Add summary to DataFrame and Vector without reportbuilder (@ananyo2012)
+  - Add support for missing data for where clause (@athityakumar)
+
+* Minor Enhancements
+  - Allow inserting or updating DataFrame vectors with single values (@baarkerlounger)
+  - Add a boolean converter to the CSV importer (@baarkerlounger)
+  - Fix documentation of replace_values method (@kojix2)
+  - Improve HTML table code of DataFrame and Vector (@Shekharrajak  )
+  - Support CSV files with empty rows (@baarkerlounger)
+  - Better DataFrame and Vector to_s methods (@baarkerlounger)
+  - Add support for histogram to Vector moving average convergence-divergence (@parthm)
+  - Add support for negative arguments to Vector.lag (@parthm)
+  - Return Nyaplot instance instead of nil for Nyaplot Vector, Category and DataFrame (@Shekharrajak)
+  - Add global configurable error stream which allows error stream to be silenced (@sivagollapalli)
+  - Rubocop update and cleanup (@zverok)
+  - Improve performance of DataFrame covariance (@genya0407)
+  - Index [] to only take index value as argument (@ananyo2012)
+  - Better error raised when Vector is missing from DataFrame (@sivagollapalli)
+  - Add default order for DataFrame (@athityakumar)
+  - Add is_values to Index (@Shekharrajak)
+  - Improve spec style in IO/SQL data source spec (@dshvimer)
+  - Open SQLite databases by bath (@dshvimer)
+  - Remove unnecessary whitespace (@Shekharrajak)
+  - Remove the .svg from Travis CI build link (@athityakumar)
+  - Fix Travis CI icon in README (@athityakumar)
+  - Replace is_nil?, not_nil? with is_values (@lokeshh)
+  - Update contributing documentation (@v0dro)
+
+* Fixes
+  - Fix missing axis labels for categorized scatter plot with Gruff (@xprazak2)
+  - Fix NMatrix Vector initialization when Vector has nils and no nm_type is given (@baarkerlounger)
+  - Fix head/tail methods on DataFrames with DateTime indexes and on Vector_at splat calls (@baarkerlounger)
+  - Fix empty DateTime Index (@zverok)
+  - Fix where clause when data contains missing/undefined values (@Shekharrajak)
+  - Fix apply_scalar_operator spec (@athityakumar)
+  - Change nil check to respond_to operator check for apply_scalar_operator (@athityakumar)
+  - Make where compatible with is_values (@athityakumar)
+  - Fix vector is_values method (@athityakumar)
+
+
 # 0.1.5 (30 January 2017)
 * Major Enhancements
   - Add Daru::Vector#group_by (@lokeshh).
@@ -19,14 +141,14 @@
   - `DataFrame#access_row` with integer index. (Yusuke Sangenya)
   - Add method alias for comparison operator. (Yusuke Sangenya)
   - Update Nokogiri version. (Yusuke Sangenya)
-  - Return `Daru::Vector` for multiple modal values for `Daru::Vector#mode`. (baarkerlounger)
+  - Return `Daru::Vector` for multiple modal values for `Daru::Vector#mode`. (@baarkerlounger)
 
 * Fixes
   - Fix many to one joins. The prior version was shifting values in the left dataframe before checking whether values in the right dataframe should be shifted.  They both need to be checked at the same time before shifting either. (@gnilrets)
   - Support formatting empty dataframes. They were returning an error before. (@gnilrets)
   - method_missing in Daru::DataFrame would not detect the correct vector if it was a String. Fixed that. (@lokeshh)
   - Fix docs of contrast_code to specify that the default value is false. (@v0dro)
-  - Fix occurence of SystemStackError due to faulty arguement passing to Array#values_at. (@v0dro)
+  - Fix occurence of SystemStackError due to faulty argument passing to Array#values_at. (@v0dro)
   - Fix `DataFrame#pivot_table` regression that raised an ArgumentError if the `:index` option was not specified. (@zverok)
   - Fix `DateFrame.rows` to accept empty argument. (@zverok)
   - Fix bug with false values on dataframe create. DataFrame from an Array of hashes wasn't being created properly when some of the values were `false`. (@gnilrets)

data/README.md

@@ -1,14 +1,35 @@
 # daru - Data Analysis in RUby
 
 [![Gem Version](https://badge.fury.io/rb/daru.svg)](http://badge.fury.io/rb/daru)
-[![Build Status](https://travis-ci.org/v0dro/daru.svg)](https://travis-ci.org/v0dro/daru)
+[![Build Status](https://travis-ci.org/SciRuby/daru.svg?branch=master)](https://travis-ci.org/SciRuby/daru)
 [![Gitter](https://badges.gitter.im/v0dro/daru.svg)](https://gitter.im/v0dro/daru?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
+[![Open Source Helpers](https://www.codetriage.com/sciruby/daru/badges/users.svg)](https://www.codetriage.com/sciruby/daru)
 
 ## Introduction
 
 daru (Data Analysis in RUby) is a library for storage, analysis, manipulation and visualization of data in Ruby.
 
-daru makes it easy and intuitive to process data predominantly through 2 data structures: `Daru::DataFrame` and `Daru::Vector`. Written in pure Ruby works with all ruby implementations. Tested with MRI 2.0, 2.1, 2.2 and 2.3.
+daru makes it easy and intuitive to process data predominantly through 2 data structures:
+`Daru::DataFrame` and `Daru::Vector`. Written in pure Ruby works with all ruby implementations.
+Tested with MRI 2.5.1 and 2.7.1.
+
+## daru plugin gems
+
+- **[daru-view](https://github.com/SciRuby/daru-view)**
+
+daru-view is for easy and interactive plotting in web application & IRuby 
+notebook. It can work in any Ruby web application frameworks like Rails, Sinatra, Nanoc and hopefully in others too.
+
+Articles/Blogs, that summarize powerful features of daru-view:
+
+* [GSoC 2017 daru-view](http://sciruby.com/blog/2017/09/01/gsoc-2017-data-visualization-using-daru-view/)
+* [GSoC 2018 Progress Report](https://github.com/SciRuby/daru-view/wiki/GSoC-2018---Progress-Report)
+* [HighCharts Official blog post regarding daru-view](https://www.highcharts.com/blog/post/i-am-ruby-developer-how-can-i-use-highcharts/)
+
+- **[daru-io](https://github.com/SciRuby/daru-io)**
+
+This gem extends support for many Import and Export methods of `Daru::DataFrame`. This gem is intended to help Rubyists who are into Data Analysis or Web Development, by serving as a general purpose conversion library that takes input in one format (say, JSON) and converts it another format (say, Avro) while also making it incredibly easy to getting started on analyzing data with daru. One can read more in [SciRuby/blog/daru-io](http://sciruby.com/blog/2017/08/29/gsoc-2017-support-to-import-export-of-more-formats/).
+
 
 ## Features
 
@@ -26,6 +47,12 @@ daru makes it easy and intuitive to process data predominantly through 2 data st
 * Quickly reducing data with pivot tables for quick data summary.
 * Import and export data from and to Excel, CSV, SQL Databases, ActiveRecord and plain text files.
 
+## Installation
+
+```console
+$ gem install daru
+```
+
 ## Notebooks
 
 #### Notebooks on most use cases
@@ -67,6 +94,7 @@ daru makes it easy and intuitive to process data predominantly through 2 data st
 * [Data Analysis in RUby: Basic data manipulation and plotting](http://v0dro.github.io/blog/2014/11/25/data-analysis-in-ruby-basic-data-manipulation-and-plotting/)
 * [Data Analysis in RUby: Splitting, sorting, aggregating data and data types](http://v0dro.github.io/blog/2015/02/24/data-analysis-in-ruby-part-2/)
 * [Finding and Combining data in daru](http://v0dro.github.io/blog/2015/08/03/finding-and-combining-data-in-daru/)
+* [Introduction to analyzing datasets with daru library](http://gafur.me/2018/02/05/analysing-datasets-with-daru-library.html)
 
 ### Time series
 
@@ -75,9 +103,9 @@ daru makes it easy and intuitive to process data predominantly through 2 data st
 
 ### Categorical Data
 
-* [Categorical Index](http://lokeshh.github.io/blog/2016/06/14/categorical-index/)
-* [Categorical Data](http://lokeshh.github.io/blog/2016/06/21/categorical-data/)
-* [Visualization with Categorical Data](http://lokeshh.github.io/blog/2016/07/02/visualization/)
+* [Categorical Index](http://lokeshh.github.io/gsoc2016/blog/2016/06/14/categorical-index/)
+* [Categorical Data](http://lokeshh.github.io/gsoc2016/blog/2016/06/21/categorical-data/)
+* [Visualization with Categorical Data](http://lokeshh.github.io/gsoc2016/blog/2016/07/02/visualization/)
 
 ## Basic Usage
 
@@ -168,7 +196,7 @@ data_frame.where(
 ```
 ![con1](images/con1.png)
 
-*Plotting* 
+*Plotting*
 
 Daru supports plotting of interactive graphs with [nyaplot](https://github.com/domitry/nyaplot). You can easily create a plot with the `#plot` method. Here we plot the gallons sold on the Y axis and name of the brand on the X axis in a bar graph.
 ``` ruby
@@ -186,17 +214,17 @@ In addition to nyaplot, daru also supports plotting out of the box with [gnuplot
 
 ## Documentation
 
-Docs can be found [here](https://rubygems.org/gems/daru).
+Docs can be found [here](http://www.rubydoc.info/gems/daru).
 
 ## Contributing
 
 Pick a feature from the Roadmap or the issue tracker or think of your own and send me a Pull Request!
 
-For details see [CONTRIBUTING](https://github.com/v0dro/daru/blob/master/CONTRIBUTING.md).
+For details see [CONTRIBUTING](https://github.com/SciRuby/daru/blob/master/CONTRIBUTING.md).
 
 ## Acknowledgements
 
-* Google and the Ruby Science Foundation for the Google Summer of Code 2016 grant for speed enhancements and implementation of support for categorical data. Special thanks to [@lokeshh](https://github.com/lokeshh), [@zverok](https://github.com/zverok) and [@agisga](https://github.com/agisga) for their efforts. 
+* Google and the Ruby Science Foundation for the Google Summer of Code 2016 grant for speed enhancements and implementation of support for categorical data. Special thanks to [@lokeshh](https://github.com/lokeshh), [@zverok](https://github.com/zverok) and [@agisga](https://github.com/agisga) for their efforts.
 * Google and the Ruby Science Foundation for the Google Summer of Code 2015 grant for further developing daru and integrating it with other ruby gems.
 * Thank you [last.fm](http://www.last.fm/) for making user data accessible to the public.
 

data/ReleasePolicy.md

@@ -0,0 +1,20 @@
+# Gem Release Policy
+
+Applicable to Daru > 0.1.6
+
+## Versioning
+
+Daru follows semantic versioning whereby the version number is always in the form MAJOR.MINOR.PATCH
+
+* Patch bump = Bug fixes
+* Minor bump = New features but backwards compatible
+* Major bump = API breaking changes
+
+For Major and Minor bumps release candidates should be released around 2 weeks prior to the bump and are indicated by MAJOR.MINOR.0.rc.
+
+For more information see the full semantic versioning specification at http://semver.org/.
+
+## Release Timing
+
+Patch releases should be done after every fix of a major bug (as tagged in the github issue tracker).
+Major releases should be kept to the minimum.