MARQ 0.0.1

data/install_scripts/GEO/series/GSE8895.yaml

@@ -0,0 +1,31 @@
+--- 
+:description: |-
+  In contrast to batch cultivation, chemostat cultivation allows the identification of carbon source responses without interference by carbon-catabolite repression, accumulation of toxic products, and differences in specific growth rate. This study focuses on the yeast Saccharomyces cerevisiae, grown in aerobic, carbon-limited chemostat cultures. Genome-wide transcript levels and in vivo fluxes were compared for growth on two sugars, glucose and maltose, and for two C2-compounds, ethanol and acetate. In contrast to previous reports on batch cultures, few genes (180 genes) responded to changes of the carbon source by a changed transcript level. Very few transcript levels were changed when glucose as the growth-limiting nutrient was compared with maltose (33 transcripts), or when acetate was compared with ethanol (16 transcripts). Although metabolic flux analysis using a stoichiometric model revealed major changes in the central carbon metabolism, only 117 genes exhibited a significantly different transcript level when sugars and C2-compounds were provided as the growthlimiting nutrient. Despite the extensive knowledge on carbon source regulation in yeast, many of the carbon source-responsive genes encoded proteins with unknown or incompletely characterized biological functions. In silico promoter analysis of carbon source-responsive genes confirmed the involvement of several known transcriptional regulators and suggested the involvement of additional regulators. Transcripts involved in the glyoxylate cycle and gluconeogenesis showed a good correlation with in vivo fluxes. This correlation was, however, not observed for other important pathways, including the pentose-phosphate pathway, tricarboxylic acid cycle, and, in particular, glycolysis. These results indicate that in vivo fluxes in the central carbon metabolism of S. cerevisiae grown in steadystate, carbon-limited chemostat cultures are controlled to a large extent via post-transcriptional mechanisms.
+  Keywords: medium composition
+:arrays: 
+  GSM225400: 
+    condition: Ethanol limited
+  GSM225401: 
+    condition: Ethanol limited
+  GSM29917: 
+    condition: Aerobic glucose limited
+  GSM225402: 
+    condition: Acetate limited
+  GSM225403: 
+    condition: Acetate limited
+  GSM225404: 
+    condition: Acetate limited
+  GSM225405: 
+    condition: Maltose limited
+  GSM225406: 
+    condition: Maltose limited
+  GSM225407: 
+    condition: Maltose limited
+  GSM29912: 
+    condition: Aerobic glucose limited
+  GSM225399: 
+    condition: Ethanol limited
+  GSM29914: 
+    condition: Aerobic glucose limited
+:title: Role of Transcriptional Regulation in Controlling Fluxes in Central Carbon Metabolism of Saccharomyces cerevisiae
+:platform: GPL90

data/install_scripts/GEO/series/GSE8897.yaml

@@ -0,0 +1,17 @@
+--- 
+:description: |-
+  Prolonged cultivation (>25 generations) of Saccharomyces cerevisiae in aerobic, maltose-limited chemostat cultures led to profound physiological changes. Maltose hypersensitivity was observed when cells from prolonged cultivations were suddenly exposed to excess maltose. This substrate hypersensitivity was evident from massive cell lysis and loss of viability. During prolonged cultivation at a fixed specific growth rate, the affinity for the growth-limiting nutrient (i.e., maltose) increased, as evident from a decreasing residual maltose concentration. Furthermore, the capacity of maltose-dependent proton uptake increased up to 2.5-fold during prolonged cultivation. Genome-wide transcriptome analysis showed that the increased maltose transport capacity was not primarily due to increased transcript levels of maltose-permease genes upon prolonged cultivation. We propose that selection for improved substrate affinity (ratio of maximum substrate consumption rate and substrate saturation constant) in maltose-limited cultures leads to selection for cells with an increased capacity for maltose uptake. At the same time, the accumulative nature of maltose-proton symport in S. cerevisiae leads to unrestricted uptake when maltose-adapted cells are exposed to a substrate excess. These changes were retained after isolation of individual cell lines from the chemostat cultures and nonselective cultivation, indicating that mutations were involved. The observed trade-off between substrate affinity and substrate tolerance may be relevant for metabolic engineering and strain selection for utilization of substrates that are taken up by proton symport.
+  Keywords: Evolution
+:arrays: 
+  GSM225461: 
+    condition: Polongued aerobic maltose limited
+  GSM225405: 
+    condition: Maltose limited
+  GSM225406: 
+    condition: Maltose limited
+  GSM225463: 
+    condition: Polongued aerobic maltose limited
+  GSM225407: 
+    condition: Maltose limited
+:title: Prolonged Maltose-Limited Cultivation of Saccharomyces cerevisiae
+:platform: GPL90

data/install_scripts/GEO/series/GSE8898.yaml

@@ -0,0 +1,18 @@
+--- 
+:description: "Prolonged cultivation of Saccharomyces cerevisiae in aerobic, glucose-limited chemostat cultures (dilution rate, 0\xC2\xB710 h\xE2\x80\x931) resulted in a progressive decrease of the residual glucose concentration (from 20 to 8 mg l\xE2\x80\x931 after 200 generations). This increase in the affinity for glucose was accompanied by a fivefold decrease of fermentative capacity, and changes in cellular morphology. These phenotypic changes were retained when single-cell isolates from prolonged cultures were used to inoculate fresh chemostat cultures, indicating that genetic changes were involved. Kinetic analysis of glucose transport in an \xE2\x80\x98evolved\xE2\x80\x99 strain revealed a decreased Km, while Vmax was slightly increased relative to the parental strain. Apparently, fermentative capacity in the evolved strain was not controlled by glucose uptake. Instead, enzyme assays in cell extracts of the evolved strain revealed strongly decreased capacities of enzymes in the lower part of glycolysis. This decrease was corroborated by genome-wide transcriptome analysis using DNA microarrays. In aerobic batch cultures on 20 g glucose l\xE2\x80\x931, the specific growth rate of the evolved strain was lower than that of the parental strain (0\xC2\xB728 and 0\xC2\xB737 h\xE2\x80\x931, respectively). Instead of the characteristic instantaneous production of ethanol that is observed when aerobic, glucose-limited cultures of wild-type S. cerevisiae are exposed to excess glucose, the evolved strain exhibited a delay of 90 min before aerobic ethanol formation set in. This study demonstrates that the effects of selection in glucose-limited chemostat cultures extend beyond glucose-transport kinetics. Although extensive physiological analysis offered insight into the underlying cellular processes, the evolutionary \xE2\x80\x98driving force\xE2\x80\x99 for several of the observed changes remains to be elucidated\n\
+  Keywords: evolution"
+:arrays: 
+  GSM225466: 
+    condition: Prologued aerobic glucose limited
+  GSM225467: 
+    condition: Prologued aerobic glucose limited
+  GSM29917: 
+    condition: Aerobic glucose limited
+  GSM225468: 
+    condition: Prologued aerobic glucose limited
+  GSM29912: 
+    condition: Aerobic glucose limited
+  GSM29914: 
+    condition: Aerobic glucose limited
+:title: Prolonged selection in aerobic, glucose-limited chemostat cultures of Saccharomyces cerevisiae
+:platform: GPL90

data/install_scripts/GEO/series/GSE8900.yaml

@@ -0,0 +1,43 @@
+--- 
+:description: |-
+  Physiological effects of carbon dioxide and impact on genome-wide transcript profiles were analysed in chemostat cultures of Saccharomyces cerevisiae. In anaerobic, glucose-limited chemostat cultures grown at atmospheric pressure, cultivation under CO2-saturated conditions had only a marginal (<10%) impact on the biomass yield. Conversely, a 25% decrease of the biomass yield was found in aerobic, glucose-limited chemostat cultures aerated with a mixture of 79% CO2 and 21% O2. This observation indicated that respiratory metabolism is more sensitive to CO2 than fermentative metabolism. Consistent with the more pronounced physiological effects of CO2 in respiratory cultures, the number of CO2-responsive transcripts was higher in aerobic cultures than in anaerobic cultures. Many genes involved in mitochondrial functions showed a transcriptional response to elevated CO2 concentrations. This is consistent with an uncoupling effect of CO2 and/or intracellular bicarbonate on the mitochondrial inner membrane. Other transcripts that showed a significant transcriptional response to elevated CO2 included NCE103 (probably encoding carbonic anhydrase), PCK1 (encoding PEP carboxykinase) and members of the IMD gene family (encoding isozymes of inosine monophosphate dehydrogenase
+  Keywords: Dose reponse
+:arrays: 
+  GSM225521: 
+    condition: Aerobic nitrogen limited chemostat culture with 79CO2
+  GSM29917: 
+    condition: Aerobic glucose limited
+  GSM225469: 
+    condition: Anaerobic glucose limited chemostat culture with 100CO2
+  GSM225514: 
+    condition: Anaerobic glucose limited chemostat culture with 100CO2
+  GSM225482: 
+    condition: Anaerobic glucose limited chemostat culture with 100CO2
+  GSM29920: 
+    condition: Aerobic nitrogen limited
+  GSM225515: 
+    condition: Aerobic glucose limited chemostat culture with 79CO2
+  GSM29921: 
+    condition: Aerobic nitrogen limited
+  GSM225516: 
+    condition: Aerobic glucose limited chemostat culture with 79CO2
+  GSM29922: 
+    condition: Aerobic nitrogen limited
+  GSM225517: 
+    condition: Aerobic glucose limited chemostat culture with 79CO2
+  GSM29934: 
+    condition: Anaerobic carbon limited
+  GSM29912: 
+    condition: Aerobic glucose limited
+  GSM225520: 
+    condition: Aerobic nitrogen limited chemostat culture with 79CO2
+  GSM225519: 
+    condition: Aerobic nitrogen limited chemostat culture with 79CO2
+  GSM29936: 
+    condition: Anaerobic carbon limited
+  GSM29914: 
+    condition: Aerobic glucose limited
+  GSM29937: 
+    condition: Anaerobic carbon limited
+:title: Genome-wide transcriptional responses of Saccharomyces cerevisiae to high carbon dioxide concentrations
+:platform: GPL90

data/install_scripts/GEO/series/GSE8982.yaml

@@ -0,0 +1,106 @@
+--- 
+:arrays: 
+  GSM227687: 
+    time: 90min
+    treatment: alpha factor 600nM
+  GSM227676: 
+    time: 30min
+    treatment: alpha factor 6nM
+  GSM227665: 
+    time: 30min
+    treatment: alpha factor 0.2nM
+  GSM227688: 
+    time: 120min
+    treatment: alpha factor 600nM
+  GSM227677: 
+    time: 60min
+    treatment: alpha factor 6nM
+  GSM227666: 
+    time: 60min
+    treatment: alpha factor 0.2nM
+  GSM227678: 
+    time: 90min
+    treatment: alpha factor 6nM
+  GSM227667: 
+    time: 90min
+    treatment: alpha factor 0.2nM
+  GSM227656: 
+    time: 5min
+    treatment: alpha factor 0.06nM
+  GSM227680: 
+    time: 15min
+    treatment: alpha factor 60nM
+  GSM227679: 
+    time: 5min
+    treatment: alpha factor 60nM
+  GSM227668: 
+    time: 120min
+    treatment: alpha factor 0.2nM
+  GSM227657: 
+    time: 15min
+    treatment: alpha factor 0.06nM
+  GSM227681: 
+    time: 30min
+    treatment: alpha factor 60nM
+  GSM227670: 
+    time: 15min
+    treatment: alpha factor 0.6nM
+  GSM227669: 
+    time: 5min
+    treatment: alpha factor 0.6nM
+  GSM227658: 
+    time: 30min
+    treatment: alpha factor 0.06nM
+  GSM227682: 
+    time: 60min
+    treatment: alpha factor 60nM
+  GSM227671: 
+    time: 30min
+    treatment: alpha factor 0.6nM
+  GSM227660: 
+    time: 90min
+    treatment: alpha factor 0.06nM
+  GSM227659: 
+    time: 60min
+    treatment: alpha factor 0.06nM
+  GSM227683: 
+    time: 90min
+    treatment: alpha factor 60nM
+  GSM227672: 
+    time: 60min
+    treatment: alpha factor 0.6nM
+  GSM227661: 
+    time: 120min
+    treatment: alpha factor 0.06nM
+  GSM227684: 
+    time: 5min
+    treatment: alpha factor 600nM
+  GSM227673: 
+    time: 90min
+    treatment: alpha factor 0.6nM
+  GSM227662: 
+    time: 150min
+    treatment: alpha factor 0.06nM
+  GSM227685: 
+    time: 30min
+    treatment: alpha factor 600nM
+  GSM227674: 
+    time: 5min
+    treatment: alpha factor 6nM
+  GSM227663: 
+    time: 5min
+    treatment: alpha factor 0.2nM
+  GSM227686: 
+    time: 60min
+    treatment: alpha factor 600nM
+  GSM227675: 
+    time: 15min
+    treatment: alpha factor 6nM
+  GSM227664: 
+    time: 15min
+    treatment: alpha factor 0.2nM
+:description: |-
+  We examined the mating response of W303 bar1delta a-type cells to six alpha-factor concentrations (0.06, 0.2, 0.6, 6, 60 and 600 nM). In each alpha factor concentration between five to seven time points were collected. The time points in all experiments (except for concentration 600 nM were time point 15 min was omitted) were 5, 15, 30, 60, and 90 min. In some of the concentrations also time points 120 and 150 min were considered. The expression of cells before and after addition of alpha-factor were compared using S. cerevisiae cDNA microarrays, in all experiments the same sample before adding alpha factor was used as a control.
+  Keywords: yeast mating response, comparison among alpha factor concentrations
+:title: Mating response - six alpha factor concentrations (0.06, 0.2, 0.6, 6, 60 and 600 nM)
+:platform: GPL5823

data/install_scripts/GEO/series/GSE920.yaml

@@ -0,0 +1,20 @@
+--- 
+:title: SUS1
+:description: |-
+  Transcriptome analysis of sus1 mutant in reference to its parental wild type reference. Iterative global median method was used for between sample normalization.
+  z-test was performed to evaluate differential gene expression.
+  Keywords: repeat sample
+:arrays: 
+  GSM13799: 
+    condition: sus1
+  GSM13800: 
+    condition: sus1
+  GSM13801: 
+    condition: sus1
+  GSM13796: 
+    condition: wt
+  GSM13797: 
+    condition: wt
+  GSM13798: 
+    condition: wt
+:platform: GPL772

data/install_scripts/GEO/series/GSE960.yaml

@@ -0,0 +1,10 @@
+--- 
+:arrays: 
+  GSM15026: 
+    treatment: wt0hRC
+  GSM15027: 
+    treatment: wt0hRC
+:description: "S. cerevisiae was grown on YEPD. For Congo Red experiments, yeast cells were grown overnight at 24 \xC2\xB0C to an optical density 0.8 - 1 (A600). The culture was refreshed to 0.2 O.D and grown at 24 \xC2\xB0C for 2h 30min. Next, culture was divided into two parts. One continues growing under same conditions (non-treated culture) while the other was supplemented with Congo Red to a final concentration of 30 \xCE\xBCg/ml. At this time cells were collected t=0h.\n\
+  Keywords: repeat sample"
+:title: wild type 0 hours exposure to congo red
+:platform: GPL920

data/install_scripts/GEO/series/GSE961.yaml

@@ -0,0 +1,14 @@
+--- 
+:arrays: 
+  GSM15015: 
+    condition: wt2hRC
+  GSM15016: 
+    condition: wt2hRC
+  GSM15017: 
+    condition: wt2hRC
+  GSM15019: 
+    condition: wt2hRC
+:description: "S. cerevisiae was grown on YEPD. For Congo Red experiments, yest cells were grown overnight at 24 \xC2\xB0C to an optical density 0.8 - 1 (A600). The culture was refreshed to 0.2 O.D and grown at 24 \xC2\xB0C for 2h 30min. Next, culture was divided into two parts. One continues growing under same conditions (non-treated culture) while the other was supplemented with Congo Red to a final concentration of 30 \xCE\xBCg/ml. Cells were collected at 2hours of growth, frozen at -80 \xC2\xB0C and processed for RNA extraction.\n\
+  Keywords: repeat sample"
+:title: wild type 2 hours exposure to congo red
+:platform: GPL920

data/install_scripts/GEO/series/GSE962.yaml

@@ -0,0 +1,20 @@
+--- 
+:arrays: 
+  GSM15055: 
+    condition: wt4hRC
+  GSM15028: 
+    condition: wt4hRC
+  GSM15053: 
+    condition: wt4hRC
+  GSM15054: 
+    condition: wt4hRC
+:description: "S. cerevisiae was grown on YEPD. For Congo Red experiments, yeast cells were grown overnight at 24 \xC2\xB0C to an optical density 0.8 - 1 (A600). The culture was refreshed to 0.2 O.D and grown at 24 \xC2\xB0C for 2h 30min. Next, culture was divided into two parts. One continues growing under same conditions (non-treated culture) while the other was supplemented with Congo Red to a final concentration of 30 \xCE\xBCg/ml. Cells were collected at 4 hours of growth, frozen at -80 \xC2\xB0C and processed for RNA extraction.\n\
+  Keywords: repeat sample"
+:title: wild type 4 hours exposure to congo red
+:platform: GPL920_GPL927
+:fields:
+  GSM15055: 'GENE._LIST'
+  GSM15028: 'GENE._LIST'
+  GSM15053: 'GENE._LIST'
+  GSM15054: 'GENE._LIST'
+  

data/install_scripts/GEO/series/GSE963.yaml

@@ -0,0 +1,14 @@
+--- 
+:arrays: 
+  GSM15022: 
+    condition: wt6hRC
+  GSM15025: 
+    condition: wt6hRC
+  GSM15052: 
+    condition: wt6hRC
+  GSM15020: 
+    condition: wt6hRC
+:description: "S. cerevisiae was grown on YEPD. For Congo Red experiments, yeast cells were grown overnight at 24 \xC2\xB0C to an optical density 0.8 - 1 (A600). The culture was refreshed to 0.2 O.D and grown at 24 \xC2\xB0C for 2h 30min. Next, culture was divided into two parts. One continues growing under same conditions (non-treated culture) while the other was supplemented with Congo Red to a final concentration of 30 \xCE\xBCg/ml. Cells were collected at 6 hours of growth, frozen at -80 \xC2\xB0C and processed for RNA extraction.\n\
+  Keywords: repeat sample"
+:title: wild type 6 hours exposure to congo red
+:platform: GPL764

data/install_scripts/GEO/series/GSE964.yaml

@@ -0,0 +1,14 @@
+--- 
+:arrays: 
+  GSM15036: 
+    condition: slt2-4hRC
+  GSM15037: 
+    condition: slt2-4hRC
+  GSM15038: 
+    condition: slt2-4hRC
+  GSM15029: 
+    condition: slt2-4hRC
+:description: "S. cerevisiae slt2\xCE\x94 strain was grown on YEPD. For Congo Red experiments, yeast cells were grown overnight at 24 \xC2\xB0C to an optical density 0.8 - 1 (A600). The culture was refreshed to 0.2 O.D and grown at 24 \xC2\xB0C for 2h 30min. Next, culture was divided into two parts. One continues growing under same conditions (non-treated culture) while the other was supplemented with Congo Red to a final concentration of 30 \xCE\xBCg/ml. Cells were collected at 4 hours of growth, frozen at -80 \xC2\xB0C and processed for RNA extraction.\n\
+  Keywords: repeat sample"
+:title: slt2 mutant 4 hours exposure to congo red
+:platform: GPL920

data/install_scripts/GEO/series/GSE965.yaml

@@ -0,0 +1,14 @@
+--- 
+:arrays: 
+  GSM15077: 
+    condition: rlm1-4hRC
+  GSM15039: 
+    condition: rlm1-4hRC
+  GSM15040: 
+    condition: rlm1-4hRC
+  GSM15076: 
+    condition: rlm1-4hRC
+:description: "S. cerevisiae rlm1\xCE\x94 strain was grown on YEPD. For Congo Red experiments, yeast cells were grown overnight at 24 \xC2\xB0C to an optical density 0.8 - 1 (A600). The culture was refreshed to 0.2 O.D and grown at 24 \xC2\xB0C for 2h 30min. Next, culture was divided into two parts. One continues growing under same conditions (non-treated culture) while the other was supplemented with Congo Red to a final concentration of 30 \xCE\xBCg/ml. Cells were collected at 4 hours of growth, frozen at -80 \xC2\xB0C and processed for RNA extraction.\n\
+  Keywords: repeat sample"
+:title: rlm1 mutant 4 hours exposure to congo red
+:platform: GPL920

data/install_scripts/GEO/series/GSE966.yaml

@@ -0,0 +1,14 @@
+--- 
+:arrays: 
+  GSM15048: 
+    condition: wt2hZYM
+  GSM15049: 
+    condition: wt2hZYM
+  GSM15050: 
+    condition: wt2hZYM
+  GSM15051: 
+    condition: wt2hZYM
+:description: "S. cerevisiae was grown on YEPD. For Zymolyase experiments, yeast cells were grown overnight at 24 \xC2\xB0C to an optical density 0.8 - 1 (A600). The culture was refreshed to 0.2 O.D and grown at 24 \xC2\xB0C for 2h 30min. Next, culture was divided into two parts. One continues growing under same conditions (non-treated culture) while the other was supplemented with 5units/ml Zymolyase 100T . Cells were collected at 2 hours of growth, frozen at -80 \xC2\xB0C and processed for RNA extraction.\n\
+  Keywords: repeat sample"
+:title: wild type two hours exposure to zymolyase
+:platform: GPL764

data/install_scripts/GEO/series/GSE993.yaml

@@ -0,0 +1,9 @@
+--- 
+:title: Universal Microarray System (UMAS) to investigate yeast response to galactose
+:description: "We describe a transcriptional analysis platform consisting of a universal micro-array system (UMAS) combined with an enzymatic manipulation step that is capable of generating expression profiles from any organism without requiring a priori species-specific sequence information. The transcriptome is converted to cDNA and processed with restriction endonucleases to generate low-complexity pools (80\xE2\x80\x93120) of equal length DNA fragments. The resulting material is amplified and detected with\n\
+  the UMAS system, comprising all possible 4,096 (4^6) DNA hexamers. Ligation to the arrays yields thousands of 14-mer sequence tags. The compendium of signals from all pools in the array-of-universal arrays comprises a full-transcriptome expression profile. The technology was validated by analysis of the galactose response of Saccharomyces cerevisiae, and the resulting profiles showed excellent agreement with the literature and real-time PCR assays.\n\
+  Keywords: other"
+:arrays: 
+  GSM15683: 
+    condition: Galactose
+:platform: GPL973

data/lib/MARQ.rb

@@ -0,0 +1,79 @@
+require 'fileutils'
+require 'yaml'
+require 'digest/md5'
+
+module MARQ
+
+  class MARQ::NoConfig < Exception; end
+
+  def self.rootdir
+    File.dirname(File.dirname(__FILE__))
+  end
+
+  config_file = [
+    File.join(ENV['HOME'], ".#{ self }"),
+    File.join(rootdir, "#{self}.config")
+  ].select{|f| File.exist? f}.first
+
+  raise MARQ::NoConfig if config_file.nil?
+
+  @@config = YAML::load(File.open(config_file))
+
+  class << self
+    %w(datadir workdir cachedir dbhost dbname dbuser dbpass).each{|dir|
+      define_method(dir, proc{@@config[dir]})
+    }
+    %w(datadir workdir cachedir).each{|dir|
+      FileUtils.mkdir_p(@@config[dir]) unless (@@config[dir].nil? || File.exists?(@@config[dir].to_s))
+    }
+
+  end
+
+end
+
+class String
+  def hash_code
+    Digest::MD5.hexdigest(self)
+  end
+end
+
+class Array
+  def count_smaller(value)
+    size = self.length
+
+    low = 0
+    last = size - 1
+    hi = size - 1
+    pos = size / 2
+    while true
+      case
+      when value == self[pos]
+        return pos 
+      when pos == last && self[pos] < value
+        return size
+      when pos == 0 && value <= self[pos] 
+        return 0 
+      when  self[pos] < value && value < self[pos + 1]
+        return pos + 1 
+      when  value < self[pos]
+        hi = pos 
+        pos = (pos - low) / 2 + low
+      when self[pos] < value
+        low = pos
+        pos = (hi - pos) / 2 + pos + 1
+      end
+    end
+  end
+end
+
+
+require 'DBcache'
+
+DBcache.config({
+  :dbuser => MARQ.dbuser,
+  :dbpass => MARQ.dbpass,
+  :dbhost => MARQ.dbhost,
+  :dbname => MARQ.dbname,
+})
+
+

data/lib/MARQ/CustomDS.rb

@@ -0,0 +1,99 @@
+require 'rbbt/util/open'
+require 'MARQ'
+require 'MARQ/ID'
+module CustomDS
+  @@r = nil
+
+  def self.customdir
+    File.join(MARQ.datadir,'CustomDS')
+  end
+
+  def self.r
+    if @@r.nil?
+      RSRuby.instance.source(MARQ.rootdir + '/R/MA.R')
+      RSRuby.instance.source(MARQ.rootdir + '/R/CustomDS.R')
+      @@r = RSRuby.instance
+    end
+    @@r
+  end
+
+  def self.info(dataset)
+    begin
+      text =  Open.read(path(dataset) + '.description')
+      if text =~ /(.*)\n--\n(.*)/
+        {:title => $1.strip, :description => $2.strip}
+      elsif text.collect.length > 1 || text.length > 200
+        {:title => "", :description => text}
+      else
+        {:title => text, :description => ""}
+      end
+    rescue Exception
+      puts $!.message
+      {:title => "" , :description => "" }
+    end
+  end
+
+  def self.path(dataset)
+    files = Dir.glob(customdir + "/*/#{ dataset }.orders")
+    if files.length == 1
+      files.first.sub(/.orders/,'')
+    else
+      Dir.glob(customdir + "/*/#{ dataset }").first
+    end
+  end
+
+  def self.organism(dataset)
+    path(dataset).match(/#{ customdir }\/(.*?)\//)[1]
+  end
+
+  def self.is_cross_platform?(dataset)
+    dataset.match(/_cross_platform/)
+  end
+
+  def self.clean(dataset)
+    dataset.sub(/_cross_platform/,'')
+  end
+  
+  def self.has_cross_platform?(dataset)
+    Dir.glob(path(clean(dataset)) + '_cross_platform.orders').any?
+  end
+
+  def self.datasets(org)
+    Dir.glob(File.join(customdir, org) + '/*.orders').collect{|f| clean(File.basename(f.sub(/.orders/,'')))}.uniq
+  end
+
+  def self.process(name)
+    puts "Processing #{ name }"
+    org = organism(name)
+    prefix = File.join(customdir, org, name)
+    conditions = Dir.glob(prefix + '/*').collect{|f| File.basename(f)} - %w(values codes info description cross_platform)
+    description = Open.read(File.join(prefix, 'description'))
+
+    info = YAML.load(File.open(File.join(prefix, 'info')))
+    r.CustomDS_process(prefix, false, conditions, description, info["two_channel"], !info["log2"])
+
+
+    codes = Open.read(File.join(prefix,'codes')).collect{|l| l.chomp}
+    cross_platform = ID.translate(org, codes) 
+
+    if cross_platform.length > codes.length / 4
+      Open.write(File.join(prefix,'cross_platform'),cross_platform.collect{|c| c || "NO MATCH"}.join("\n"))
+      r.CustomDS_process(prefix, true, conditions, description, info["two_channel"], !info["log2"])
+    end
+  end
+
+end
+
+
+if __FILE__ == $0
+  p CustomDS::datasets('sgd')
+  p CustomDS::path('HaploidData')
+  p CustomDS::path('HaploidData_cross_platform')
+  exit
+
+  org = 'sgd'
+  process = Dir.glob(File.join(CustomDS::customdir, org) + '/*').select{|f| File.directory? f}.collect{|f| File.basename(f)} - CustomDS.datasets('sgd')
+  p process
+  process.each{|d| CustomDS::process(d)}
+
+end