datafarming 2.3.2 → 3.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 1ef168cea50204b80c99a99cf3f38f358a0d84ef06de0df706cf7e07f2a42a83
4
- data.tar.gz: efd748c56269ff27569a943762ff2af940fc6b06efc7eac0a574c542ce1a7968
3
+ metadata.gz: 67ecc038ff0d8fd2e83ef1d0db8715bbec3395e3b234c96e54ed22fb4bba595d
4
+ data.tar.gz: 083b7b5911b44da84796f8b3f8005d762f091799ab7978fc95e536f324fb2cac
5
5
  SHA512:
6
- metadata.gz: 993ce76465c47dc73da8632e36e34dee5480bec25e69482d8204702e5ffae4762d631cce32f642c1a12eeb9f51f86697505762a97131387442cf5a665e0988a2
7
- data.tar.gz: 12863b1bf2ec5f348666840257fadf42ec597bc4c1671db3d957d454f265a794a1db1600eb38df206a5cbd7004ff1a122e848b91aa22ea5a2f2a069fe19f9831
6
+ metadata.gz: 981413887afe289ae19c3ce2f11b4c9bd59326bce95019c0441b0090912250c34527c96ba013a1a3e38b858cc1f8ee05636c41cda436ed436b46afd5575b7a4e
7
+ data.tar.gz: 1d796a86fed0ef6666130ef388bd0a8aa5099e6a21d527326b7218d70c1f7ee60a5b80788980fdb56ea543d18b95b616b89e1772d1cf250a34890b7873a7daf2
data/LICENSE.md CHANGED
@@ -1,7 +1,7 @@
1
1
 
2
2
  The MIT License (MIT)
3
3
 
4
- Copyright (c) 2017-2024 Paul J. Sanchez
4
+ Copyright (c) 2017-2026 Paul J. Sanchez
5
5
 
6
6
  Permission is hereby granted, free of charge, to any person obtaining a copy
7
7
  of this software and associated documentation files (the "Software"), to deal
data/README.md CHANGED
@@ -1,25 +1,29 @@
1
1
  ### SETUP
2
2
 
3
- This gem requires Ruby v3 or later to be installed on your system.
3
+ This gem requires Ruby v2.6 or later to be installed on your system. Ruby v4 or higher is strongly recommended, offering security updates along with better speed and feature improvements.
4
4
 
5
- - On MacOS, Ruby comes pre-installed. Note: You can use the homebrew package manager from [brew.sh](https://brew.sh) if you want to install a newer version of Ruby which has numerous advantages over the one that currently ships with MacOS.
5
+ - On MacOS, Ruby comes pre-installed, but is many years out-of-date at v2.6.10. Note: You can use package managers such as [mise](https://mise.jdx.dev) or [homebrew](https://brew.sh) to install the latest version of Ruby.
6
6
 
7
7
  - Windows users should use one of the installer packages available from [rubyinstaller.org](http://rubyinstaller.org/). You will need to determine whether your have a 32 or 64 bit processor, and download the appropriate installer. When running the installer, check the checkboxes to add the Ruby installation to your PATH and to associate `.rb` and `.rbw` files with Ruby. A terminal window will pop up during the installation—press "enter" to automatically install all of the required tools, and when prompted again (several minutes later) press "enter" again to end the installation.
8
8
 
9
9
  You can check that Ruby is properly installed by opening `Terminal.app` (MacOS) or `CMD.EXE` (Windows) and typing `ruby --version` (note: two dashes). You should see a response telling you which version of Ruby you are running.
10
10
 
11
- After your Ruby installation is confirmed you can install the data farming Ruby scripts by running the following command.
11
+ After your Ruby installation is confirmed, if you have an active network connection you can install the data farming Ruby scripts by running the following command.
12
12
 
13
13
  gem install datafarming
14
14
 
15
- This assumes that you have a network connection. Additional dependencies will be installed automatically. On MacOS or Linux systems, you may be required to use the `sudo` command to authenticate the installation. In that case, type `sudo gem install datafarming` and enter your password when prompted.
15
+ Additional dependencies will be installed automatically. On MacOS or Linux systems, you may be required to use the `sudo` command to validate and complete a system-wide installation. In that case, type `sudo gem install datafarming` and enter your password when prompted.
16
16
 
17
- Gem installation only needs to be done once. If you explicitly downloaded the `.gem` file rather than using a network installation, you can delete it after performing the installation. If a newer version of the gem is released, you can upgrade to it with `gem update`.
17
+ We don't generally recommend doing the following, but if all else fails you can perform a user-owned installation with the command
18
18
 
19
- ### USAGE
19
+ gem install datafarming --user-install
20
+
21
+ The specific location of the installation will depend on your OS. You will need to read documentation to identify the destination folder and manually add it to your runtime environment's search PATH.
20
22
 
21
- Ruby is a powerful and concise object-oriented scripting language. Normally, Ruby scripts are run from a command-line or terminal environment by typing the `ruby` command followed by a script name, often followed by one or more command-line arguments. However, scripts installed as gems do not need the `ruby` command to be typed explicitly. For example, typing `stripheaderdups.rb my_file.txt` will invoke the `stripheaderdups.rb` script and apply it to file `my_file.txt` in your current working directory. Note that on Windows the `.rb` suffix is not needed to run your scripts, e.g., `stripheaderdups my_file.txt` will suffice.
23
+ Gem installation only needs to be done once. If you explicitly downloaded the `.gem` file rather than using a network installation, you can delete it after performing the installation. When a newer version of the gem is released, you can upgrade to it by running `gem update`.
24
+
25
+ ### USAGE
22
26
 
23
- All scripts in this distribution are self documenting if run with a `--help`, or `-h` option. You can run the command `datafarming.rb` at any point after installing this gem to see descriptions of the various data farming utilities included in thie distribution.
27
+ Ruby is a powerful and concise object-oriented scripting language. Ruby scripts are normally run from a command-line or terminal environment by typing the `ruby` command followed by a script name, often followed by one or more command-line arguments. However, scripts installed as gems do not need the `ruby` command to be typed explicitly. For example, typing `stripheaderdups.rb my_file.txt` will invoke the `stripheaderdups.rb` script and apply it to file `my_file.txt` in your current working directory. Additionally, on Windows the `.rb` suffix is not needed to run your scripts, e.g., `stripheaderdups my_file.txt` will suffice.
24
28
 
25
- After installing the gem and making sure that gems are on your PATH, run the command `datafarming.rb` for a summary/overview of the various data farming tools.
29
+ All scripts in this distribution are self documenting if invoked with a `--help`, or `-h` option. After installing the gem and making sure that gems are on your PATH, run the command `datafarming.rb` at any time to see short overview descriptions of the various data farming tools provided in this distribution.
data/datafarming.gemspec CHANGED
@@ -7,9 +7,9 @@ Gem::Specification.new do |s|
7
7
  s.name = "datafarming"
8
8
  s.version = DataFarming::VERSION
9
9
  s.date = "#{Date.today}"
10
- s.summary = "Useful scripts for data farming."
10
+ s.summary = "Useful tools for data farming."
11
11
  s.homepage = "https://bitbucket.org/paul_j_sanchez/datafarmingrubyscripts"
12
- s.email = "pjs@alum.mit.edu"
12
+ s.email = "prof.pjs@icloud.com"
13
13
  s.description = "Ruby scripts for data farming, including pre- and" \
14
14
  "post-processing, design generation and scaling, and run control."
15
15
  s.author = "Paul J Sanchez"
data/exe/datafarming.rb CHANGED
@@ -22,25 +22,25 @@ lines = [
22
22
  ' • blank2csv.rb —',
23
23
  ' convert whitespace-delimited files to Comma Separated Values (CSV) files',
24
24
  ' • cat.rb —',
25
- ' Unix cat program work-alike for Windows—for viewing or creating text files',
26
- ' or concatenating multiple files, based on stdio',
25
+ ' Unix \'cat\' program work-alike for Windows—for viewing or creating text files',
26
+ ' or concatenating multiple files, based on stdio.',
27
27
  ' • convert_line_endings.rb —',
28
28
  ' convert text files (including CSV files) to your current operating system',
29
- ' environment (e.g., DOS to Unix or Unix to DOS)',
29
+ ' environment (e.g., DOS to Unix or Unix to DOS).',
30
30
  ' • csv2blank.rb —',
31
- ' convert CSV files to whitespace-delimited files',
31
+ ' convert CSV files to whitespace-delimited files.',
32
32
  ' • pool_files.rb —',
33
33
  ' pool columns from separate files into a new file, useful for combining',
34
- ' design file & output files or multiple output files',
34
+ ' design file with output files or multiple output files.',
35
35
  ' • stripheaders.rb , stripheaderdups.rb —',
36
36
  ' remove column headers from a file, or duplicate headers within a file,',
37
- ' respectively',
37
+ ' respectively.',
38
38
  '',
39
39
  '#### DESIGN GENERATORS',
40
40
  '',
41
41
  ' • cross.rb —',
42
42
  ' create a combinatorial design by crossing all combinations of any # of',
43
- ' individual smaller designs',
43
+ ' individual smaller designs.',
44
44
  ' • generate_design.rb —',
45
45
  ' allowable design types are',
46
46
  ' - resv : Resolution V fractional factorial at 2 levels per factor;',
@@ -57,21 +57,25 @@ lines = [
57
57
  ' allow partial correlation amongst the interactions which',
58
58
  ' significantly reduces the number of design points;',
59
59
  ' all of which generate designs with stacking, replication, and individualized',
60
- ' factor scaling',
60
+ ' factor scaling. For continuous-factor designs (fbd, fbs, nolh) individual',
61
+ ' factors can be converted to discrete with some potential loss of',
62
+ ' orthogonality, or nearly-balanced categoricals where category counts',
63
+ ' differ by no more than 1.',
64
+ ' Note - use caution with multiple categoricals, since not all possible',
65
+ ' combinations may be realized.',
61
66
  '',
62
67
  '#### RUN CONTROL',
63
68
  '',
64
69
  ' • batchrunner.rb —',
65
- ' run a model interactively with replication',
70
+ ' run a model interactively with replication.',
66
71
  ' • rundesign_general.rb —',
67
- ' run control to replicate a designed experiment applied to a model',
72
+ ' run control to replicate a designed experiment applied to a model.',
73
+ ' • run_yaml_design.rb —',
74
+ ' run control to replicate a designed experiment applied to a model that',
75
+ ' uses YAML for inputs.',
68
76
  '',
69
77
  '#### CONFIDENCE INTERVALS FOR TIME SERIES',
70
78
  '',
71
- ' • mser.rb —',
72
- ' use MSER truncation to remove initial transient effects for time-series',
73
- ' output, reporting truncated average and number of observations for each run',
74
- ' to facilitate construction of a properly weighted confidence interval.',
75
79
  ' • mser_nolbm.rb —',
76
80
  ' use MSER truncation to remove initial transient effects for time-series',
77
81
  ' output, then calculate a 95% confidence interval on the remaining data using',
@@ -1,9 +1,5 @@
1
- #!/usr/bin/env ruby -w
2
- # frozen_string_literal: true
1
+ #!/usr/bin/env ruby
3
2
 
4
- # require 'colorize'
5
- # String.disable_colorization false
6
- # require 'datafarming/error_handling'
7
3
  RubyVM::YJIT.enable if defined?(RubyVM::YJIT) && RubyVM::YJIT.respond_to?(:enable)
8
4
 
9
5
  require 'optparse'
@@ -11,12 +7,24 @@ require 'yaml'
11
7
 
12
8
  require 'datafarming/scale_design'
13
9
 
14
- DESIGN_TYPES = %i[resv ccd rotatable nolh fbd fbs].freeze
15
- REQUIRED_SPECS = %i[min max].freeze
16
- ELECTIVE_SPECS = %i[id decimals].freeze
17
- LEVELS = [17, 33, 65, 129, 257, 512].freeze
10
+ CLASSICAL_DESIGNS = %i[resv ccd rotatable nolh].freeze
11
+ FREQ_DESIGNS = %i[fbd fbs].freeze
12
+ DESIGN_TYPES = (CLASSICAL_DESIGNS + FREQ_DESIGNS).freeze
13
+ MIN_MAX = %i[min max].freeze
14
+ ELECTIVE_SPECS = %i[id decimals cat disc].freeze
15
+ NOLH_LEVELS = [17, 33, 65, 129, 257, 512].freeze
18
16
  TWO_PI = 2.0 * Math::PI
19
17
 
18
+ DEFAULT_OPTIONS = {
19
+ design: nil,
20
+ levels: nil,
21
+ headers: true,
22
+ replicate: 1,
23
+ stack: 1,
24
+ centerdup: false,
25
+ specs: []
26
+ }
27
+
20
28
  HELP_MSG = [
21
29
  'Generate, scale, stack, and replicate a variety of statistical designs',
22
30
  "of experiments. You are required to specify the desired '--design TYPE'.",
@@ -25,7 +33,7 @@ HELP_MSG = [
25
33
  ].freeze
26
34
 
27
35
  EXTENDED_HELP = [
28
- "====================\n",
36
+ # "====================\n",
29
37
  "Using the '-k' option will specify the number of factors for a standardized",
30
38
  'design, in which all factors are normalized to the range of [-1, 1].',
31
39
  '',
@@ -35,19 +43,47 @@ EXTENDED_HELP = [
35
43
  "specifications provided (thus overriding the '-k' option). Each factor's",
36
44
  'specification is separated from the others by whitespace.',
37
45
  '',
38
- 'Factor specifications contain two to four comma-separated components, and',
39
- 'cannot include any whitespace. The format is:',
46
+ 'Continuous factor specifications contain two to four comma-separated',
47
+ 'components, and cannot include any whitespace. The format is:',
40
48
  "\n id:STRING,min:NUMBER,max:NUMBER,decimals:INTEGER\n",
41
- "Components can be arranged in any order. The components 'min' and 'max'",
49
+ "Components can be provided in any order. The components 'min' and 'max'",
42
50
  'are required, and define the range of experimentation for this factor.',
43
- "Components 'id' and 'decimals' are optional. If 'id' is omitted, id",
44
- "values will be generated for use with the '--headers' option. If 'decimals'",
45
- 'is omitted, the factors values will be displayed with default precision.',
46
- ' '
51
+ "Components 'id' and 'decimals' are optional. If 'id' is omitted, id values",
52
+ "will be generated for use with the '--headers' option. If 'decimals' is",
53
+ 'omitted, the values will be displayed with default (full) precision.',
54
+ '',
55
+ 'Discrete and categorical factors are also options. The core design',
56
+ 'underpinning mixtures of continuous/discrete/categorical should be one',
57
+ "of the continuous designs: fbd, fbs, or nolh",
58
+ '',
59
+ "Discrete factors are generated using a specification of the form:",
60
+ "\n disc:INTEGER,id:STRING,min:NUMBER,max:NUMBER,decimals:INTEGER\n",
61
+ 'The first INTEGER specifies the number of discrete levels. As with',
62
+ "continuous factors, 'min' and 'max' are required, 'id' and 'decimals'",
63
+ "are optional. The results can be non-integer if 'min' and 'max' are",
64
+ 'specified with decimal values.',
65
+ '',
66
+ "Categorical factors are generated using a specification of the form:",
67
+ "\n cat:RANGE,id:STRING\n",
68
+ 'or',
69
+ "\n cat:'[ARRAY]',id:STRING\n",
70
+ "If the RANGE option is used, it is a Ruby range of the form 'FIRST..LAST'",
71
+ "or 'FIRST...LAST'. The two-dot version generates the sequence of values",
72
+ 'between FIRST and LAST, inclusive, while the three-dot version excludes the',
73
+ 'LAST value. Ruby allows ranges of integers or lexicographic strings. If the',
74
+ 'ARRAY specification is used, a comma-separated set of specific values is',
75
+ 'enumerated. Note that the square brackets surrounding the array values MUST',
76
+ 'be contained between single quotation marks.',
77
+ '',
78
+ "For large designs with many factors, using the '-k' and '-o' options",
79
+ 'together is highly recommended. The resulting YAML output can then be',
80
+ 'edited at your leisure, while a long multi-factor set of specification',
81
+ 'strings is prone to human error on entry.',
82
+ '',
47
83
  ].freeze
48
84
 
49
85
  EXAMPLES = [
50
- "\n====================\n",
86
+ # "\n====================\n",
51
87
  'EXAMPLE 1:',
52
88
  "\n #{File.basename($PROGRAM_NAME)} --design fbd -k 3 --no-headers",
53
89
  "\n Generate a standardized frequency-based design with three factors,",
@@ -55,7 +91,7 @@ EXAMPLES = [
55
91
  '',
56
92
  'EXAMPLE 2:',
57
93
  "\n #{File.basename($PROGRAM_NAME)} -d nolh id:Fred,min:-10,max:10 decimals:5,max:100,min:0",
58
- "\n Generate a NOLH for two factors. The first factor will have a column",
94
+ "\n Generate an NOLH for two factors. The first factor will have a column",
59
95
  " header of 'Fred', and values will be scaled in the range -10..10. The",
60
96
  " second column will have an automatically generated header 'X_002', and",
61
97
  ' values will be in the range 0..100 with five decimal places of precision.',
@@ -63,15 +99,23 @@ EXAMPLES = [
63
99
  " by whitespace.",
64
100
  '',
65
101
  'EXAMPLE 3:',
66
- "\n #{File.basename($PROGRAM_NAME)} -d nolh -k 5 --output_yaml > my_template.yml",
67
- "\n Generate a YAML template for a 5-factor NOLH design, and redirect it",
102
+ "\n #{File.basename($PROGRAM_NAME)} -d fbd disc:5,min:-1.0,max:1.0 cat:a..e",
103
+ "\n Generate a mixed discrete/categorical design based on an underlying",
104
+ ' frequency-based design. The first factor will have values stepped between',
105
+ ' -1 and 1 in steps of 0.5. The second column will consist of a balanced mix',
106
+ ' of the letters a, b, c, d, and e. Both columns will have automatically',
107
+ ' generated headers.',
108
+ '',
109
+ 'EXAMPLE 4:',
110
+ "\n #{File.basename($PROGRAM_NAME)} -d nolh -k 5 -o my_template.yml",
111
+ "\n Generate a YAML template for a 5-factor NOLH design, and send it",
68
112
  " to the file 'my_template.yml'. (The output file can be given any name",
69
113
  ' of your choosing.) The resulting YAML file has an intuitive human',
70
114
  " readable format. It can be edited using your favorite programmer's",
71
115
  ' text editor (NOTE: word processors will not work!). After saving your',
72
116
  ' revisions, you can read it back in to generate the actual design using',
73
117
  " the '--input_yaml' option:\n",
74
- " #{File.basename($PROGRAM_NAME)} --input_yaml my_template.yml",
118
+ " #{File.basename($PROGRAM_NAME)} -i my_template.yml",
75
119
  ' '
76
120
  ].freeze
77
121
 
@@ -81,13 +125,13 @@ def prog_help(opts)
81
125
  end
82
126
 
83
127
  def verbose(opts)
84
- puts opts
128
+ # puts opts
85
129
  puts EXTENDED_HELP.join("\n")
86
130
  exit
87
131
  end
88
132
 
89
133
  def examples(opts)
90
- puts opts
134
+ # puts opts
91
135
  puts EXAMPLES.join("\n")
92
136
  exit
93
137
  end
@@ -97,6 +141,53 @@ def fatal_err(msg)
97
141
  exit 1
98
142
  end
99
143
 
144
+ def thresholds(k)
145
+ const = Math::PI / (2 * k)
146
+ Array.new(k) { |idx| Math.sin(const * (2 * (idx+1) - k)) }
147
+ end
148
+
149
+ class Array
150
+ @@toggle = true
151
+ def toggle_cmp(x, y)
152
+ if x == 0.0
153
+ (@@toggle ^= true) ? x > y : x >= y
154
+ else
155
+ x >= y
156
+ end
157
+ end
158
+
159
+ def discretize(spec)
160
+ stepsize = (spec[:max] - spec[:min]) / (spec[:disc] - 1)
161
+ ary = Array.new(spec[:disc]) { |i| spec[:min] + i * stepsize }
162
+ if FREQ_DESIGNS.include? spec[:design]
163
+ t = thresholds(spec[:disc])
164
+ else
165
+ stepsize = 2.0 / spec[:disc]
166
+ t = Array.new(spec[:disc] - 1) { |i| (i + 1) * stepsize - 1 } + [1]
167
+ end
168
+ decimals = spec[:decimals] || 15
169
+ self.map { |value| ary[t.bsearch_index { |x| x >= value }].round(decimals) }
170
+ end
171
+
172
+ def categorize(spec)
173
+ categories = if spec[:cat].include? '..'
174
+ Range.new(*(spec[:cat].split('..'))).to_a
175
+ elsif spec[:cat].is_a? Array
176
+ spec[:cat]
177
+ else
178
+ fail "Invalid category specification: #{spec[:cat]}"
179
+ end
180
+ # t = thresholds(categories.size)
181
+ if FREQ_DESIGNS.include? spec[:design]
182
+ t = thresholds(categories.size)
183
+ else
184
+ stepsize = 2.0 / categories.size
185
+ t = Array.new(categories.size - 1) { |i| (i + 1) * stepsize - 1 } + [1]
186
+ end
187
+ self.map { |x| categories[t.bsearch_index { |entry| entry >= x }] }
188
+ end
189
+ end
190
+
100
191
  def resv(options)
101
192
  require 'datafarming/res_v_seqs'
102
193
  num_factors = options[:specs].size
@@ -155,12 +246,10 @@ def freqs2cols(options, design_set)
155
246
  freqs = ds.freqs[design_num]
156
247
  Array.new(freqs.size) do |row|
157
248
  omega = freqs[row]
158
- v = Array.new(ds.nyq) do |i|
249
+ Array.new(ds.nyq) do |i|
159
250
  f = ((i * omega) % ds.nyq).to_f / ds.nyq
160
251
  Math.sin(f * TWO_PI)
161
252
  end
162
- scale_factor = v.max
163
- v.map { |x| (x / scale_factor).round(15) }
164
253
  end.transpose
165
254
  end
166
255
 
@@ -175,6 +264,7 @@ def fbs(options)
175
264
  end
176
265
 
177
266
  def validate_opts(options)
267
+ options.delete_if { |k, v| v.nil? }
178
268
  required_options = [:design]
179
269
  missing = required_options - options.keys
180
270
  fatal_err("Missing required options: --#{missing.join(', --')}") unless missing.empty?
@@ -182,18 +272,18 @@ def validate_opts(options)
182
272
  end
183
273
 
184
274
  def validate(spec)
185
- unknown = spec.keys - REQUIRED_SPECS - ELECTIVE_SPECS
275
+ unknown = spec.keys - MIN_MAX - ELECTIVE_SPECS
186
276
  fatal_err("Unknown factor spec(s): #{unknown}") unless unknown.empty?
187
- missing = REQUIRED_SPECS - spec.keys
277
+ missing = (has_cat = spec.keys.include?(:cat)) ? [] : MIN_MAX - spec.keys
188
278
  fatal_err("Factor spec #{spec} missing #{missing}") unless missing.empty?
189
279
  end
190
280
 
191
281
  def optional_specs(factor_spec, i)
192
- YAML.safe_load("{#{factor_spec}}".gsub(':', ': '),
193
- permitted_classes: [Symbol]).tap do |spec|
282
+ YAML.safe_load("{#{factor_spec}}".gsub(':', ': '), permitted_classes: [Symbol]).tap do |spec|
194
283
  spec.transform_keys!(&:to_sym)
195
- spec[:decimals] ||= nil
196
284
  spec[:id] ||= "X_#{format('%03d', (i + 1))}"
285
+ spec.delete_if { |k, v| v.nil? }
286
+ spec[:decimals] = nil unless (spec.has_key?(:decimals) || spec.has_key?(:cat))
197
287
  validate(spec)
198
288
  end
199
289
  end
@@ -207,20 +297,12 @@ def parse_specs(options, remainder)
207
297
  end
208
298
 
209
299
  def parse_args
210
- default_options = {
211
- headers: true,
212
- replicate: 1,
213
- stack: 1,
214
- center: false,
215
- specs: []
216
- }
217
-
218
300
  yaml_input = false
219
301
  if ARGV.size > 2 && ARGV.any? { |elt| elt.eql?("-i") || elt.eql?("--input_yaml") }
220
302
  STDERR.puts 'NOTE: All other arguments ignored when using YAML input'
221
303
  end
222
304
 
223
- options = default_options
305
+ options = DEFAULT_OPTIONS
224
306
 
225
307
  begin
226
308
  remainder = OptionParser.new do |opts|
@@ -243,25 +325,48 @@ def parse_args
243
325
  'Replicate QTY times', '(default: 1)')
244
326
  opts.on('-s', '--stack QTY', /[1-9][0-9]*/, Integer,
245
327
  'Stack QTY times', '(default: 1)')
246
- opts.on('--[no-]center', 'NOTE: Applies only to stacked designs',
247
- 'Stacking includes center pt','(default: no-center)')
328
+ opts.on('-c', '--[no-]centerdup', 'NOTE: Applies only to stacked designs',
329
+ 'Stacking can duplicate the center point','(default: no-centerdup)')
248
330
  opts.on('--[no-]headers', 'Output has column headers', '(default: headers)')
249
- opts.on('-l', '--levels QTY', Regexp.new("(#{LEVELS.join(')|(')})"),
331
+ opts.on('-l', '--levels QTY', Regexp.new("(#{NOLH_LEVELS.join(')|(')})"),
250
332
  Integer, 'NOTE: Applies only to NOLH designs',
251
333
  'Number of levels in the base NOLH:',
252
- " #{LEVELS.join ' '}")
253
- opts.on('-o', '--output_yaml', 'Write design YAML to STDOUT')
254
- opts.on("-i PATH", "--input_yaml", 'Read design YAML from PATH',
255
- 'Mutually exclusive with all other options, prints',
256
- 'warning to STDERR if other options are passed') do |path|
334
+ " #{NOLH_LEVELS.join ' '}")
335
+ opts.on('-o', '--output_yaml [FILENAME]',
336
+ 'Write design YAML to FILENAME (or STDOUT)'
337
+ ) do |fname|
338
+ if fname.nil? || '-' == fname
339
+ STDOUT
340
+ else
341
+ fname
342
+ end
343
+ end
344
+ opts.on(
345
+ "-i", "--input_yaml [FILENAME]",
346
+ 'Read design YAML from FILENAME (or STDIN)',
347
+ '(Mutually exclusive with all other options,',
348
+ 'prints warning to STDERR if other options',
349
+ 'are passed)'
350
+ ) do |fname|
257
351
  ARGV.clear
258
- options = {specs: []}
352
+ # options = {specs: []}
353
+ options = DEFAULT_OPTIONS
259
354
  yaml_input = true
260
- YAML::safe_load_file(path, permitted_classes: [Symbol]).each do |k, v|
261
- options.merge!({k.to_sym => v})
355
+ if fname.nil? || '-' == fname
356
+ YAML::safe_load(
357
+ STDIN.read, permitted_classes: [Symbol]
358
+ ).each do |k, v|
359
+ options[k.to_sym] = v
360
+ end
361
+ else
362
+ YAML::safe_load_file(
363
+ fname, permitted_classes: [Symbol]
364
+ ).each do |k, v|
365
+ options[k.to_sym] = v
366
+ end
262
367
  end
263
368
  end
264
- end.parse!(into: options)
369
+ end.parse!(into: options)
265
370
  rescue OptionParser::ParseError => error
266
371
  fatal_err("#{error}\n(-h or --help will show valid options)")
267
372
  end
@@ -280,23 +385,23 @@ def parse_args
280
385
  options
281
386
  end
282
387
 
283
- def stack(design, num_stacks, centered)
388
+ def stack(design, num_stacks, center_pt_dup)
284
389
  return design if num_stacks < 2
285
- fatal_err("Stacking #{num_stacks} times exceeds number of design columns") if num_stacks > design[0].length
390
+ fatal_err("Stacking #{num_stacks} times exceeds number of design columns") if num_stacks > design.size
286
391
  num_stacks -= 1
287
- result = design.transpose
392
+ result = design
288
393
  if num_stacks > 0
289
- design_t = if centered
290
- design.transpose
394
+ design_t = if center_pt_dup
395
+ design.clone
291
396
  else
292
- design.delete_if {|row| row.all? &:zero? }.transpose
397
+ design.transpose.delete_if {|row| row.all? &:zero? }.transpose
293
398
  end
294
399
  num_stacks.times do
295
400
  design_t.rotate!
296
- result.map!.with_index { |line, i| line + design_t[i] }
401
+ result.size.times { |i| result[i] += design_t[i] }
297
402
  end
298
403
  end
299
- result.transpose
404
+ result
300
405
  end
301
406
 
302
407
  ARGV << '--help' if ARGV.empty?
@@ -304,26 +409,46 @@ ARGV << '--help' if ARGV.empty?
304
409
  options = parse_args
305
410
 
306
411
  if options[:output_yaml]
307
- options.delete :output_yaml
308
- puts options.to_yaml
412
+ o_file = (STDOUT == options[:output_yaml]) ? STDOUT : File.open(options[:output_yaml], 'w')
413
+ options.delete(:output_yaml)
414
+ begin
415
+ o_file.puts options.to_yaml
416
+ ensure
417
+ o_file.close unless STDIN == o_file
418
+ end
309
419
  else
310
- base_design = method(options[:design]).call(options)
420
+ base_design = method(options[:design]).call(options).transpose.first(options[:specs].size)
311
421
  separator = ','
312
422
  puts Array.new(options[:specs].size) { |i| options[:specs][i][:id] }.join(separator) if options[:headers]
313
423
 
314
424
  # stack
315
- design = stack(base_design, options[:stack], options[:center])
316
- .transpose.first(options[:specs].size).transpose
425
+ design = stack(base_design, options[:stack], options[:centerdup])
317
426
 
318
427
  # scale
319
- scaler = options[:specs].map do |spec|
320
- Scaler.new(min: spec[:min], max: spec[:max], decimals: spec[:decimals])
428
+ scale_factor = design.map &:max
429
+ d_type = options[:design]
430
+ design.map!.with_index do |v, i|
431
+ break if i >= options[:specs].size
432
+ spec = options[:specs][i]
433
+ if spec[:cat]
434
+ spec[:design] = d_type
435
+ v.categorize(spec)
436
+ elsif spec[:disc]
437
+ spec[:design] = d_type
438
+ v.discretize(spec)
439
+ else
440
+ scaler = Scaler.new(spec)
441
+ v.map do |x|
442
+ scaler.scale((x / scale_factor[i]).round(15))
443
+ end
444
+ end
321
445
  end
322
446
 
323
447
  # replicate
448
+ result = design.transpose
324
449
  options[:replicate].times do
325
- design.each do |line|
326
- puts line.map.with_index { |v, i| scaler[i].scale(v) }.join(separator)
450
+ result.each do |line|
451
+ puts line.join(separator)
327
452
  end
328
453
  end
329
454
  end
data/exe/mser_nolbm.rb CHANGED
@@ -1,5 +1,7 @@
1
1
  #!/usr/bin/env ruby -w
2
2
 
3
+ RubyVM::YJIT.enable if defined?(RubyVM::YJIT) && RubyVM::YJIT.respond_to?(:enable)
4
+
3
5
  require 'rubygems' if RUBY_VERSION =~ /^1\.8/
4
6
  require 'colorize'
5
7
 
@@ -84,19 +86,19 @@ def nolbm(data)
84
86
  avg = best[1]
85
87
  index = best[2]
86
88
  length = data.length - index
87
- b = [length / 30, length / index].min
89
+ b = (index > 0) ? [length / 30, length / index].min : length / 30
88
90
  m, excess = length.divmod b
89
- incr, discard = excess.divmod b
90
- m += incr
91
+ incr, discard = excess.divmod m
92
+ b += incr
91
93
  index += discard
92
- usable = data.drop(index).each_slice(m).to_a
93
- batch_means = Array.new(b) do |i|
94
+ usable = data.drop(index).each_slice(b).to_a
95
+ batch_means = Array.new(m) do |i|
94
96
  QuickStats.new.add_set(usable[i]).avg
95
97
  end
96
98
  sum_squared_deviations = batch_means
97
99
  .map { |y| square(y - avg)}
98
100
  .inject(&:+)
99
- df = (b - 1)
101
+ df = (m - 1)
100
102
  var_hat = sum_squared_deviations / df
101
103
  std_dev = Math.sqrt(var_hat)
102
104
  std_err = Math.sqrt(var_hat / b)
data/exe/mser_olbm.rb CHANGED
@@ -1,5 +1,7 @@
1
1
  #!/usr/bin/env ruby -w
2
2
 
3
+ RubyVM::YJIT.enable if defined?(RubyVM::YJIT) && RubyVM::YJIT.respond_to?(:enable)
4
+
3
5
  require 'rubygems' if RUBY_VERSION =~ /^1\.8/
4
6
  require 'colorize'
5
7