datafarming 2.3.1 → 3.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: e3cb821ea35c4864abcf73017e1d62c13a776584d0ce0861189fe100ae30977d
4
- data.tar.gz: bf152bb6bc1626368c47cd04ee698bf4c5ad9ec0a0c45388b18e3141fadccc91
3
+ metadata.gz: 67ecc038ff0d8fd2e83ef1d0db8715bbec3395e3b234c96e54ed22fb4bba595d
4
+ data.tar.gz: 083b7b5911b44da84796f8b3f8005d762f091799ab7978fc95e536f324fb2cac
5
5
  SHA512:
6
- metadata.gz: a65d0a1d9c2b9c82ae33b879d5ebe37ea079c5502da4dfb19975688d8643b4b35237447e7ea0efd3384c83591579e2aad6b417f71e644d1911058ad0b68f7ccb
7
- data.tar.gz: fa8f9738a009069a8b812c08ffed22aae198348001f487962056f16b17192a8f8316e837b3931bd47d759b1b7f8951784725811621bfa1d403bb91651e7d4836
6
+ metadata.gz: 981413887afe289ae19c3ce2f11b4c9bd59326bce95019c0441b0090912250c34527c96ba013a1a3e38b858cc1f8ee05636c41cda436ed436b46afd5575b7a4e
7
+ data.tar.gz: 1d796a86fed0ef6666130ef388bd0a8aa5099e6a21d527326b7218d70c1f7ee60a5b80788980fdb56ea543d18b95b616b89e1772d1cf250a34890b7873a7daf2
data/LICENSE.md CHANGED
@@ -1,7 +1,7 @@
1
1
 
2
2
  The MIT License (MIT)
3
3
 
4
- Copyright (c) 2017-2024 Paul J. Sanchez
4
+ Copyright (c) 2017-2026 Paul J. Sanchez
5
5
 
6
6
  Permission is hereby granted, free of charge, to any person obtaining a copy
7
7
  of this software and associated documentation files (the "Software"), to deal
data/README.md CHANGED
@@ -1,25 +1,29 @@
1
1
  ### SETUP
2
2
 
3
- This gem requires Ruby v3 or later to be installed on your system.
3
+ This gem requires Ruby v2.6 or later to be installed on your system. Ruby v4 or higher is strongly recommended, offering security updates along with better speed and feature improvements.
4
4
 
5
- - On MacOS, Ruby comes pre-installed. Note: You can use the homebrew package manager from [brew.sh](https://brew.sh) if you want to install a newer version of Ruby which has numerous advantages over the one that currently ships with MacOS.
5
+ - On MacOS, Ruby comes pre-installed, but is many years out-of-date at v2.6.10. Note: You can use package managers such as [mise](https://mise.jdx.dev) or [homebrew](https://brew.sh) to install the latest version of Ruby.
6
6
 
7
7
  - Windows users should use one of the installer packages available from [rubyinstaller.org](http://rubyinstaller.org/). You will need to determine whether your have a 32 or 64 bit processor, and download the appropriate installer. When running the installer, check the checkboxes to add the Ruby installation to your PATH and to associate `.rb` and `.rbw` files with Ruby. A terminal window will pop up during the installation—press "enter" to automatically install all of the required tools, and when prompted again (several minutes later) press "enter" again to end the installation.
8
8
 
9
9
  You can check that Ruby is properly installed by opening `Terminal.app` (MacOS) or `CMD.EXE` (Windows) and typing `ruby --version` (note: two dashes). You should see a response telling you which version of Ruby you are running.
10
10
 
11
- After your Ruby installation is confirmed you can install the data farming Ruby scripts by running the following command.
11
+ After your Ruby installation is confirmed, if you have an active network connection you can install the data farming Ruby scripts by running the following command.
12
12
 
13
13
  gem install datafarming
14
14
 
15
- This assumes that you have a network connection. Additional dependencies will be installed automatically. On MacOS or Linux systems, you may be required to use the `sudo` command to authenticate the installation. In that case, type `sudo gem install datafarming` and enter your password when prompted.
15
+ Additional dependencies will be installed automatically. On MacOS or Linux systems, you may be required to use the `sudo` command to validate and complete a system-wide installation. In that case, type `sudo gem install datafarming` and enter your password when prompted.
16
16
 
17
- Gem installation only needs to be done once. If you explicitly downloaded the `.gem` file rather than using a network installation, you can delete it after performing the installation. If a newer version of the gem is released, you can upgrade to it with `gem update`.
17
+ We don't generally recommend doing the following, but if all else fails you can perform a user-owned installation with the command
18
18
 
19
- ### USAGE
19
+ gem install datafarming --user-install
20
+
21
+ The specific location of the installation will depend on your OS. You will need to read documentation to identify the destination folder and manually add it to your runtime environment's search PATH.
20
22
 
21
- Ruby is a powerful and concise object-oriented scripting language. Normally, Ruby scripts are run from a command-line or terminal environment by typing the `ruby` command followed by a script name, often followed by one or more command-line arguments. However, scripts installed as gems do not need the `ruby` command to be typed explicitly. For example, typing `stripheaderdups.rb my_file.txt` will invoke the `stripheaderdups.rb` script and apply it to file `my_file.txt` in your current working directory. Note that on Windows the `.rb` suffix is not needed to run your scripts, e.g., `stripheaderdups my_file.txt` will suffice.
23
+ Gem installation only needs to be done once. If you explicitly downloaded the `.gem` file rather than using a network installation, you can delete it after performing the installation. When a newer version of the gem is released, you can upgrade to it by running `gem update`.
24
+
25
+ ### USAGE
22
26
 
23
- All scripts in this distribution are self documenting if run with a `--help`, or `-h` option. You can run the command `datafarming.rb` at any point after installing this gem to see descriptions of the various data farming utilities included in thie distribution.
27
+ Ruby is a powerful and concise object-oriented scripting language. Ruby scripts are normally run from a command-line or terminal environment by typing the `ruby` command followed by a script name, often followed by one or more command-line arguments. However, scripts installed as gems do not need the `ruby` command to be typed explicitly. For example, typing `stripheaderdups.rb my_file.txt` will invoke the `stripheaderdups.rb` script and apply it to file `my_file.txt` in your current working directory. Additionally, on Windows the `.rb` suffix is not needed to run your scripts, e.g., `stripheaderdups my_file.txt` will suffice.
24
28
 
25
- After installing the gem and making sure that gems are on your PATH, run the command `datafarming.rb` for a summary/overview of the various data farming tools.
29
+ All scripts in this distribution are self documenting if invoked with a `--help`, or `-h` option. After installing the gem and making sure that gems are on your PATH, run the command `datafarming.rb` at any time to see short overview descriptions of the various data farming tools provided in this distribution.
data/datafarming.gemspec CHANGED
@@ -7,9 +7,9 @@ Gem::Specification.new do |s|
7
7
  s.name = "datafarming"
8
8
  s.version = DataFarming::VERSION
9
9
  s.date = "#{Date.today}"
10
- s.summary = "Useful scripts for data farming."
10
+ s.summary = "Useful tools for data farming."
11
11
  s.homepage = "https://bitbucket.org/paul_j_sanchez/datafarmingrubyscripts"
12
- s.email = "pjs@alum.mit.edu"
12
+ s.email = "prof.pjs@icloud.com"
13
13
  s.description = "Ruby scripts for data farming, including pre- and" \
14
14
  "post-processing, design generation and scaling, and run control."
15
15
  s.author = "Paul J Sanchez"
data/exe/datafarming.rb CHANGED
@@ -22,25 +22,25 @@ lines = [
22
22
  ' • blank2csv.rb —',
23
23
  ' convert whitespace-delimited files to Comma Separated Values (CSV) files',
24
24
  ' • cat.rb —',
25
- ' Unix cat program work-alike for Windows—for viewing or creating text files',
26
- ' or concatenating multiple files, based on stdio',
25
+ ' Unix \'cat\' program work-alike for Windows—for viewing or creating text files',
26
+ ' or concatenating multiple files, based on stdio.',
27
27
  ' • convert_line_endings.rb —',
28
28
  ' convert text files (including CSV files) to your current operating system',
29
- ' environment (e.g., DOS to Unix or Unix to DOS)',
29
+ ' environment (e.g., DOS to Unix or Unix to DOS).',
30
30
  ' • csv2blank.rb —',
31
- ' convert CSV files to whitespace-delimited files',
31
+ ' convert CSV files to whitespace-delimited files.',
32
32
  ' • pool_files.rb —',
33
33
  ' pool columns from separate files into a new file, useful for combining',
34
- ' design file & output files or multiple output files',
34
+ ' design file with output files or multiple output files.',
35
35
  ' • stripheaders.rb , stripheaderdups.rb —',
36
36
  ' remove column headers from a file, or duplicate headers within a file,',
37
- ' respectively',
37
+ ' respectively.',
38
38
  '',
39
39
  '#### DESIGN GENERATORS',
40
40
  '',
41
41
  ' • cross.rb —',
42
42
  ' create a combinatorial design by crossing all combinations of any # of',
43
- ' individual smaller designs',
43
+ ' individual smaller designs.',
44
44
  ' • generate_design.rb —',
45
45
  ' allowable design types are',
46
46
  ' - resv : Resolution V fractional factorial at 2 levels per factor;',
@@ -57,21 +57,25 @@ lines = [
57
57
  ' allow partial correlation amongst the interactions which',
58
58
  ' significantly reduces the number of design points;',
59
59
  ' all of which generate designs with stacking, replication, and individualized',
60
- ' factor scaling',
60
+ ' factor scaling. For continuous-factor designs (fbd, fbs, nolh) individual',
61
+ ' factors can be converted to discrete with some potential loss of',
62
+ ' orthogonality, or nearly-balanced categoricals where category counts',
63
+ ' differ by no more than 1.',
64
+ ' Note - use caution with multiple categoricals, since not all possible',
65
+ ' combinations may be realized.',
61
66
  '',
62
67
  '#### RUN CONTROL',
63
68
  '',
64
69
  ' • batchrunner.rb —',
65
- ' run a model interactively with replication',
70
+ ' run a model interactively with replication.',
66
71
  ' • rundesign_general.rb —',
67
- ' run control to replicate a designed experiment applied to a model',
72
+ ' run control to replicate a designed experiment applied to a model.',
73
+ ' • run_yaml_design.rb —',
74
+ ' run control to replicate a designed experiment applied to a model that',
75
+ ' uses YAML for inputs.',
68
76
  '',
69
77
  '#### CONFIDENCE INTERVALS FOR TIME SERIES',
70
78
  '',
71
- ' • mser.rb —',
72
- ' use MSER truncation to remove initial transient effects for time-series',
73
- ' output, reporting truncated average and number of observations for each run',
74
- ' to facilitate construction of a properly weighted confidence interval.',
75
79
  ' • mser_nolbm.rb —',
76
80
  ' use MSER truncation to remove initial transient effects for time-series',
77
81
  ' output, then calculate a 95% confidence interval on the remaining data using',
@@ -1,21 +1,30 @@
1
- #!/usr/bin/env ruby -w
2
- # frozen_string_literal: true
1
+ #!/usr/bin/env ruby
3
2
 
4
- # require 'colorize'
5
- # String.disable_colorization false
6
- # require 'datafarming/error_handling'
3
+ RubyVM::YJIT.enable if defined?(RubyVM::YJIT) && RubyVM::YJIT.respond_to?(:enable)
7
4
 
8
5
  require 'optparse'
9
6
  require 'yaml'
10
7
 
11
8
  require 'datafarming/scale_design'
12
9
 
13
- DESIGN_TYPES = %i[resv ccd rotatable nolh fbd fbs].freeze
14
- REQUIRED_SPECS = %i[min max].freeze
15
- ELECTIVE_SPECS = %i[id decimals].freeze
16
- LEVELS = [17, 33, 65, 129, 257, 512].freeze
10
+ CLASSICAL_DESIGNS = %i[resv ccd rotatable nolh].freeze
11
+ FREQ_DESIGNS = %i[fbd fbs].freeze
12
+ DESIGN_TYPES = (CLASSICAL_DESIGNS + FREQ_DESIGNS).freeze
13
+ MIN_MAX = %i[min max].freeze
14
+ ELECTIVE_SPECS = %i[id decimals cat disc].freeze
15
+ NOLH_LEVELS = [17, 33, 65, 129, 257, 512].freeze
17
16
  TWO_PI = 2.0 * Math::PI
18
17
 
18
+ DEFAULT_OPTIONS = {
19
+ design: nil,
20
+ levels: nil,
21
+ headers: true,
22
+ replicate: 1,
23
+ stack: 1,
24
+ centerdup: false,
25
+ specs: []
26
+ }
27
+
19
28
  HELP_MSG = [
20
29
  'Generate, scale, stack, and replicate a variety of statistical designs',
21
30
  "of experiments. You are required to specify the desired '--design TYPE'.",
@@ -24,7 +33,7 @@ HELP_MSG = [
24
33
  ].freeze
25
34
 
26
35
  EXTENDED_HELP = [
27
- "====================\n",
36
+ # "====================\n",
28
37
  "Using the '-k' option will specify the number of factors for a standardized",
29
38
  'design, in which all factors are normalized to the range of [-1, 1].',
30
39
  '',
@@ -34,19 +43,47 @@ EXTENDED_HELP = [
34
43
  "specifications provided (thus overriding the '-k' option). Each factor's",
35
44
  'specification is separated from the others by whitespace.',
36
45
  '',
37
- 'Factor specifications contain two to four comma-separated components, and',
38
- 'cannot include any whitespace. The format is:',
46
+ 'Continuous factor specifications contain two to four comma-separated',
47
+ 'components, and cannot include any whitespace. The format is:',
39
48
  "\n id:STRING,min:NUMBER,max:NUMBER,decimals:INTEGER\n",
40
- "Components can be arranged in any order. The components 'min' and 'max'",
49
+ "Components can be provided in any order. The components 'min' and 'max'",
41
50
  'are required, and define the range of experimentation for this factor.',
42
- "Components 'id' and 'decimals' are optional. If 'id' is omitted, id",
43
- "values will be generated for use with the '--headers' option. If 'decimals'",
44
- 'is omitted, the factors values will be displayed with default precision.',
45
- ' '
51
+ "Components 'id' and 'decimals' are optional. If 'id' is omitted, id values",
52
+ "will be generated for use with the '--headers' option. If 'decimals' is",
53
+ 'omitted, the values will be displayed with default (full) precision.',
54
+ '',
55
+ 'Discrete and categorical factors are also options. The core design',
56
+ 'underpinning mixtures of continuous/discrete/categorical should be one',
57
+ "of the continuous designs: fbd, fbs, or nolh",
58
+ '',
59
+ "Discrete factors are generated using a specification of the form:",
60
+ "\n disc:INTEGER,id:STRING,min:NUMBER,max:NUMBER,decimals:INTEGER\n",
61
+ 'The first INTEGER specifies the number of discrete levels. As with',
62
+ "continuous factors, 'min' and 'max' are required, 'id' and 'decimals'",
63
+ "are optional. The results can be non-integer if 'min' and 'max' are",
64
+ 'specified with decimal values.',
65
+ '',
66
+ "Categorical factors are generated using a specification of the form:",
67
+ "\n cat:RANGE,id:STRING\n",
68
+ 'or',
69
+ "\n cat:'[ARRAY]',id:STRING\n",
70
+ "If the RANGE option is used, it is a Ruby range of the form 'FIRST..LAST'",
71
+ "or 'FIRST...LAST'. The two-dot version generates the sequence of values",
72
+ 'between FIRST and LAST, inclusive, while the three-dot version excludes the',
73
+ 'LAST value. Ruby allows ranges of integers or lexicographic strings. If the',
74
+ 'ARRAY specification is used, a comma-separated set of specific values is',
75
+ 'enumerated. Note that the square brackets surrounding the array values MUST',
76
+ 'be contained between single quotation marks.',
77
+ '',
78
+ "For large designs with many factors, using the '-k' and '-o' options",
79
+ 'together is highly recommended. The resulting YAML output can then be',
80
+ 'edited at your leisure, while a long multi-factor set of specification',
81
+ 'strings is prone to human error on entry.',
82
+ '',
46
83
  ].freeze
47
84
 
48
85
  EXAMPLES = [
49
- "\n====================\n",
86
+ # "\n====================\n",
50
87
  'EXAMPLE 1:',
51
88
  "\n #{File.basename($PROGRAM_NAME)} --design fbd -k 3 --no-headers",
52
89
  "\n Generate a standardized frequency-based design with three factors,",
@@ -54,7 +91,7 @@ EXAMPLES = [
54
91
  '',
55
92
  'EXAMPLE 2:',
56
93
  "\n #{File.basename($PROGRAM_NAME)} -d nolh id:Fred,min:-10,max:10 decimals:5,max:100,min:0",
57
- "\n Generate a NOLH for two factors. The first factor will have a column",
94
+ "\n Generate an NOLH for two factors. The first factor will have a column",
58
95
  " header of 'Fred', and values will be scaled in the range -10..10. The",
59
96
  " second column will have an automatically generated header 'X_002', and",
60
97
  ' values will be in the range 0..100 with five decimal places of precision.',
@@ -62,15 +99,23 @@ EXAMPLES = [
62
99
  " by whitespace.",
63
100
  '',
64
101
  'EXAMPLE 3:',
65
- "\n #{File.basename($PROGRAM_NAME)} -d nolh -k 5 --output_yaml > my_template.yml",
66
- "\n Generate a YAML template for a 5-factor NOLH design, and redirect it",
102
+ "\n #{File.basename($PROGRAM_NAME)} -d fbd disc:5,min:-1.0,max:1.0 cat:a..e",
103
+ "\n Generate a mixed discrete/categorical design based on an underlying",
104
+ ' frequency-based design. The first factor will have values stepped between',
105
+ ' -1 and 1 in steps of 0.5. The second column will consist of a balanced mix',
106
+ ' of the letters a, b, c, d, and e. Both columns will have automatically',
107
+ ' generated headers.',
108
+ '',
109
+ 'EXAMPLE 4:',
110
+ "\n #{File.basename($PROGRAM_NAME)} -d nolh -k 5 -o my_template.yml",
111
+ "\n Generate a YAML template for a 5-factor NOLH design, and send it",
67
112
  " to the file 'my_template.yml'. (The output file can be given any name",
68
113
  ' of your choosing.) The resulting YAML file has an intuitive human',
69
114
  " readable format. It can be edited using your favorite programmer's",
70
115
  ' text editor (NOTE: word processors will not work!). After saving your',
71
116
  ' revisions, you can read it back in to generate the actual design using',
72
117
  " the '--input_yaml' option:\n",
73
- " #{File.basename($PROGRAM_NAME)} --input_yaml my_template.yml",
118
+ " #{File.basename($PROGRAM_NAME)} -i my_template.yml",
74
119
  ' '
75
120
  ].freeze
76
121
 
@@ -80,13 +125,13 @@ def prog_help(opts)
80
125
  end
81
126
 
82
127
  def verbose(opts)
83
- puts opts
128
+ # puts opts
84
129
  puts EXTENDED_HELP.join("\n")
85
130
  exit
86
131
  end
87
132
 
88
133
  def examples(opts)
89
- puts opts
134
+ # puts opts
90
135
  puts EXAMPLES.join("\n")
91
136
  exit
92
137
  end
@@ -96,6 +141,53 @@ def fatal_err(msg)
96
141
  exit 1
97
142
  end
98
143
 
144
+ def thresholds(k)
145
+ const = Math::PI / (2 * k)
146
+ Array.new(k) { |idx| Math.sin(const * (2 * (idx+1) - k)) }
147
+ end
148
+
149
+ class Array
150
+ @@toggle = true
151
+ def toggle_cmp(x, y)
152
+ if x == 0.0
153
+ (@@toggle ^= true) ? x > y : x >= y
154
+ else
155
+ x >= y
156
+ end
157
+ end
158
+
159
+ def discretize(spec)
160
+ stepsize = (spec[:max] - spec[:min]) / (spec[:disc] - 1)
161
+ ary = Array.new(spec[:disc]) { |i| spec[:min] + i * stepsize }
162
+ if FREQ_DESIGNS.include? spec[:design]
163
+ t = thresholds(spec[:disc])
164
+ else
165
+ stepsize = 2.0 / spec[:disc]
166
+ t = Array.new(spec[:disc] - 1) { |i| (i + 1) * stepsize - 1 } + [1]
167
+ end
168
+ decimals = spec[:decimals] || 15
169
+ self.map { |value| ary[t.bsearch_index { |x| x >= value }].round(decimals) }
170
+ end
171
+
172
+ def categorize(spec)
173
+ categories = if spec[:cat].include? '..'
174
+ Range.new(*(spec[:cat].split('..'))).to_a
175
+ elsif spec[:cat].is_a? Array
176
+ spec[:cat]
177
+ else
178
+ fail "Invalid category specification: #{spec[:cat]}"
179
+ end
180
+ # t = thresholds(categories.size)
181
+ if FREQ_DESIGNS.include? spec[:design]
182
+ t = thresholds(categories.size)
183
+ else
184
+ stepsize = 2.0 / categories.size
185
+ t = Array.new(categories.size - 1) { |i| (i + 1) * stepsize - 1 } + [1]
186
+ end
187
+ self.map { |x| categories[t.bsearch_index { |entry| entry >= x }] }
188
+ end
189
+ end
190
+
99
191
  def resv(options)
100
192
  require 'datafarming/res_v_seqs'
101
193
  num_factors = options[:specs].size
@@ -154,12 +246,10 @@ def freqs2cols(options, design_set)
154
246
  freqs = ds.freqs[design_num]
155
247
  Array.new(freqs.size) do |row|
156
248
  omega = freqs[row]
157
- v = Array.new(ds.nyq) do |i|
249
+ Array.new(ds.nyq) do |i|
158
250
  f = ((i * omega) % ds.nyq).to_f / ds.nyq
159
251
  Math.sin(f * TWO_PI)
160
252
  end
161
- scale_factor = v.max
162
- v.map { |x| (x / scale_factor).round(15) }
163
253
  end.transpose
164
254
  end
165
255
 
@@ -174,6 +264,7 @@ def fbs(options)
174
264
  end
175
265
 
176
266
  def validate_opts(options)
267
+ options.delete_if { |k, v| v.nil? }
177
268
  required_options = [:design]
178
269
  missing = required_options - options.keys
179
270
  fatal_err("Missing required options: --#{missing.join(', --')}") unless missing.empty?
@@ -181,18 +272,18 @@ def validate_opts(options)
181
272
  end
182
273
 
183
274
  def validate(spec)
184
- unknown = spec.keys - REQUIRED_SPECS - ELECTIVE_SPECS
275
+ unknown = spec.keys - MIN_MAX - ELECTIVE_SPECS
185
276
  fatal_err("Unknown factor spec(s): #{unknown}") unless unknown.empty?
186
- missing = REQUIRED_SPECS - spec.keys
277
+ missing = (has_cat = spec.keys.include?(:cat)) ? [] : MIN_MAX - spec.keys
187
278
  fatal_err("Factor spec #{spec} missing #{missing}") unless missing.empty?
188
279
  end
189
280
 
190
281
  def optional_specs(factor_spec, i)
191
- YAML.safe_load("{#{factor_spec}}".gsub(':', ': '),
192
- permitted_classes: [Symbol]).tap do |spec|
282
+ YAML.safe_load("{#{factor_spec}}".gsub(':', ': '), permitted_classes: [Symbol]).tap do |spec|
193
283
  spec.transform_keys!(&:to_sym)
194
- spec[:decimals] ||= nil
195
284
  spec[:id] ||= "X_#{format('%03d', (i + 1))}"
285
+ spec.delete_if { |k, v| v.nil? }
286
+ spec[:decimals] = nil unless (spec.has_key?(:decimals) || spec.has_key?(:cat))
196
287
  validate(spec)
197
288
  end
198
289
  end
@@ -206,20 +297,12 @@ def parse_specs(options, remainder)
206
297
  end
207
298
 
208
299
  def parse_args
209
- default_options = {
210
- headers: true,
211
- replicate: 1,
212
- stack: 1,
213
- center: false,
214
- specs: []
215
- }
216
-
217
300
  yaml_input = false
218
301
  if ARGV.size > 2 && ARGV.any? { |elt| elt.eql?("-i") || elt.eql?("--input_yaml") }
219
302
  STDERR.puts 'NOTE: All other arguments ignored when using YAML input'
220
303
  end
221
304
 
222
- options = default_options
305
+ options = DEFAULT_OPTIONS
223
306
 
224
307
  begin
225
308
  remainder = OptionParser.new do |opts|
@@ -242,25 +325,48 @@ def parse_args
242
325
  'Replicate QTY times', '(default: 1)')
243
326
  opts.on('-s', '--stack QTY', /[1-9][0-9]*/, Integer,
244
327
  'Stack QTY times', '(default: 1)')
245
- opts.on('--[no-]center', 'NOTE: Applies only to stacked designs',
246
- 'Stacking includes center pt','(default: no-center)')
328
+ opts.on('-c', '--[no-]centerdup', 'NOTE: Applies only to stacked designs',
329
+ 'Stacking can duplicate the center point','(default: no-centerdup)')
247
330
  opts.on('--[no-]headers', 'Output has column headers', '(default: headers)')
248
- opts.on('-l', '--levels QTY', Regexp.new("(#{LEVELS.join(')|(')})"),
331
+ opts.on('-l', '--levels QTY', Regexp.new("(#{NOLH_LEVELS.join(')|(')})"),
249
332
  Integer, 'NOTE: Applies only to NOLH designs',
250
333
  'Number of levels in the base NOLH:',
251
- " #{LEVELS.join ' '}")
252
- opts.on('-o', '--output_yaml', 'Write design YAML to STDOUT')
253
- opts.on("-i PATH", "--input_yaml", 'Read design YAML from PATH',
254
- 'Mutually exclusive with all other options, prints',
255
- 'warning to STDERR if other options are passed') do |path|
334
+ " #{NOLH_LEVELS.join ' '}")
335
+ opts.on('-o', '--output_yaml [FILENAME]',
336
+ 'Write design YAML to FILENAME (or STDOUT)'
337
+ ) do |fname|
338
+ if fname.nil? || '-' == fname
339
+ STDOUT
340
+ else
341
+ fname
342
+ end
343
+ end
344
+ opts.on(
345
+ "-i", "--input_yaml [FILENAME]",
346
+ 'Read design YAML from FILENAME (or STDIN)',
347
+ '(Mutually exclusive with all other options,',
348
+ 'prints warning to STDERR if other options',
349
+ 'are passed)'
350
+ ) do |fname|
256
351
  ARGV.clear
257
- options = {specs: []}
352
+ # options = {specs: []}
353
+ options = DEFAULT_OPTIONS
258
354
  yaml_input = true
259
- YAML::safe_load_file(path, permitted_classes: [Symbol]).each do |k, v|
260
- options.merge!({k.to_sym => v})
355
+ if fname.nil? || '-' == fname
356
+ YAML::safe_load(
357
+ STDIN.read, permitted_classes: [Symbol]
358
+ ).each do |k, v|
359
+ options[k.to_sym] = v
360
+ end
361
+ else
362
+ YAML::safe_load_file(
363
+ fname, permitted_classes: [Symbol]
364
+ ).each do |k, v|
365
+ options[k.to_sym] = v
366
+ end
261
367
  end
262
368
  end
263
- end.parse!(into: options)
369
+ end.parse!(into: options)
264
370
  rescue OptionParser::ParseError => error
265
371
  fatal_err("#{error}\n(-h or --help will show valid options)")
266
372
  end
@@ -279,23 +385,23 @@ def parse_args
279
385
  options
280
386
  end
281
387
 
282
- def stack(design, num_stacks, centered)
388
+ def stack(design, num_stacks, center_pt_dup)
283
389
  return design if num_stacks < 2
284
- fatal_err("Stacking #{num_stacks} times exceeds number of design columns") if num_stacks > design[0].length
390
+ fatal_err("Stacking #{num_stacks} times exceeds number of design columns") if num_stacks > design.size
285
391
  num_stacks -= 1
286
- result = design.transpose
392
+ result = design
287
393
  if num_stacks > 0
288
- design_t = if centered
289
- design.transpose
394
+ design_t = if center_pt_dup
395
+ design.clone
290
396
  else
291
- design.delete_if {|row| row.all? &:zero? }.transpose
397
+ design.transpose.delete_if {|row| row.all? &:zero? }.transpose
292
398
  end
293
399
  num_stacks.times do
294
400
  design_t.rotate!
295
- result.map!.with_index { |line, i| line + design_t[i] }
401
+ result.size.times { |i| result[i] += design_t[i] }
296
402
  end
297
403
  end
298
- result.transpose
404
+ result
299
405
  end
300
406
 
301
407
  ARGV << '--help' if ARGV.empty?
@@ -303,26 +409,46 @@ ARGV << '--help' if ARGV.empty?
303
409
  options = parse_args
304
410
 
305
411
  if options[:output_yaml]
306
- options.delete :output_yaml
307
- puts options.to_yaml
412
+ o_file = (STDOUT == options[:output_yaml]) ? STDOUT : File.open(options[:output_yaml], 'w')
413
+ options.delete(:output_yaml)
414
+ begin
415
+ o_file.puts options.to_yaml
416
+ ensure
417
+ o_file.close unless STDIN == o_file
418
+ end
308
419
  else
309
- base_design = method(options[:design]).call(options)
420
+ base_design = method(options[:design]).call(options).transpose.first(options[:specs].size)
310
421
  separator = ','
311
422
  puts Array.new(options[:specs].size) { |i| options[:specs][i][:id] }.join(separator) if options[:headers]
312
423
 
313
424
  # stack
314
- design = stack(base_design, options[:stack], options[:center])
315
- .transpose.first(options[:specs].size).transpose
425
+ design = stack(base_design, options[:stack], options[:centerdup])
316
426
 
317
427
  # scale
318
- scaler = options[:specs].map do |spec|
319
- Scaler.new(min: spec[:min], max: spec[:max], decimals: spec[:decimals])
428
+ scale_factor = design.map &:max
429
+ d_type = options[:design]
430
+ design.map!.with_index do |v, i|
431
+ break if i >= options[:specs].size
432
+ spec = options[:specs][i]
433
+ if spec[:cat]
434
+ spec[:design] = d_type
435
+ v.categorize(spec)
436
+ elsif spec[:disc]
437
+ spec[:design] = d_type
438
+ v.discretize(spec)
439
+ else
440
+ scaler = Scaler.new(spec)
441
+ v.map do |x|
442
+ scaler.scale((x / scale_factor[i]).round(15))
443
+ end
444
+ end
320
445
  end
321
446
 
322
447
  # replicate
448
+ result = design.transpose
323
449
  options[:replicate].times do
324
- design.each do |line|
325
- puts line.map.with_index { |v, i| scaler[i].scale(v) }.join(separator)
450
+ result.each do |line|
451
+ puts line.join(separator)
326
452
  end
327
453
  end
328
454
  end
data/exe/mser_nolbm.rb CHANGED
@@ -1,5 +1,7 @@
1
1
  #!/usr/bin/env ruby -w
2
2
 
3
+ RubyVM::YJIT.enable if defined?(RubyVM::YJIT) && RubyVM::YJIT.respond_to?(:enable)
4
+
3
5
  require 'rubygems' if RUBY_VERSION =~ /^1\.8/
4
6
  require 'colorize'
5
7
 
@@ -84,35 +86,31 @@ def nolbm(data)
84
86
  avg = best[1]
85
87
  index = best[2]
86
88
  length = data.length - index
87
- m = [length / 31, index].max
88
- b = length / m
89
- m = length / b
90
- index += length - m * b
91
-
92
- mean_stats = QuickStats.new
93
- batch_means = Array.new(b) do
94
- mean_stats.reset
95
- m.times do
96
- mean_stats.new_obs(data[index])
97
- index += 1
98
- end
99
- mean_stats.avg
89
+ b = (index > 0) ? [length / 30, length / index].min : length / 30
90
+ m, excess = length.divmod b
91
+ incr, discard = excess.divmod m
92
+ b += incr
93
+ index += discard
94
+ usable = data.drop(index).each_slice(b).to_a
95
+ batch_means = Array.new(m) do |i|
96
+ QuickStats.new.add_set(usable[i]).avg
100
97
  end
101
- sum_squared_deviations = batch_means.
102
- map { |y| square(y - avg)}.
103
- inject(&:+)
104
- var_hat = sum_squared_deviations / (b - 1)
105
- df = (b - 1)
98
+ sum_squared_deviations = batch_means
99
+ .map { |y| square(y - avg)}
100
+ .inject(&:+)
101
+ df = (m - 1)
102
+ var_hat = sum_squared_deviations / df
103
+ std_dev = Math.sqrt(var_hat)
106
104
  std_err = Math.sqrt(var_hat / b)
107
105
  half_width = T_VALUE[df] * std_err
108
106
  lower = avg - half_width
109
107
  upper = avg + half_width
110
- printf "%f,%f,%d,%f,%f\n", avg, std_err, df, lower, upper
108
+ printf "%f,%f,%f,%d,%f,%f\n", avg, std_dev, std_err, df, lower, upper
111
109
  end
112
110
 
113
- puts "sample_mean,std_err,df,lower95_bound,upper95_bound"
114
- if ARGF.filename == "-"
115
- data = STDIN.readlines
111
+ puts "sample_mean,std_dev,std_err,df,lower95_bound,upper95_bound"
112
+ if ARGV.empty?
113
+ data = ARGF.readlines
116
114
  nolbm(data)
117
115
  else
118
116
  ARGV.each do |fname|