aldrich 0.1.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (3) hide show
  1. checksums.yaml +7 -0
  2. data/README.md +39 -0
  3. metadata +111 -0
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: f9ad3dbae285308fa95e947f5d7450a11925b8e3
4
+ data.tar.gz: 8e97692597e8839f2a72937b5b9ff714fb333b00
5
+ SHA512:
6
+ metadata.gz: 8935dc5dd69305250610036d32cbed16f487656fa8766337a46b30de4222272fdd1ffab8d8b996e6768280eb558c5408c91f4bcd912c135465fd0234219a67ef
7
+ data.tar.gz: 82156b34177b25159df7443039a715b463f5693af3acf0c8b939db815d6bf16736553e64c1dab312cb2b2cd3e5349e4ad6997172e80b90dde63e341c3fe64a9f
@@ -0,0 +1,39 @@
1
+ # aldrich
2
+
3
+ A project that is an attempt to rehabilitate and generalize a data ingestion/importation service originally built for a retail, e-commerce platform. It aims to provide a robust solution for mapping intermediate representations of database records (CSV at the moment) to persisted data. The process allows simulation, validation, and control of the arrangement of the final data form. The ultimate goal is a project that is flexible enough to use in a pure-Ruby, Rails, or standalone application backed by data storage of your choice (as long as model representations can be accessed via Ruby).
4
+
5
+ Currently, there's not much here. This document will be updated with friendlier bits once the project is close to usable. Until then, what follows is disjointed bits of the original README file that still mostly apply.
6
+
7
+ # Process Workflow
8
+
9
+ This section contain general explanation for the internal implementation of the data compliation process. It includes flowchats outlining each level of the process. Not every tiny detail is included here as the service itself is rather complex and expansive.
10
+
11
+ Much more specific documentation as to API, purpose, and rationale is provided inside each ruby code file in the form of Tomdoc comments.
12
+
13
+ ## General Importer Process
14
+
15
+ The top-most level of the Importer service workflow is reasonably straightforward. Simulation is mandatory when imports are performed via the Importer::Processors::Import class. The overall workflow can accept data from the web form or a RESTful request.
16
+
17
+ ![Alt text](/lib/aldrich/doc/importer_process.png?raw=true "General Importer Process")
18
+
19
+ ## Operation Group Handling
20
+
21
+ The primary iteration during an import is performed on each group code. Group codes are typically the SKU used for a Product, wether it parents Variants or not. All operations will be placed into separate sets by the group code and each invocation of Importer internals will operate on that set of Operation objects.
22
+
23
+ Generally speaking, error catching is done for each operation group. All other errors raised at this time will be considered unhandled, exceptional behavior. The structure of the service is such that the code used between simulations and imports differs as little as is possible, to allow simulations to be reasonable representations of the actual import process.
24
+
25
+ Currently simulation is unable to validate data against database-level validations. This results in some small circumstances where code validations raise no errors, but attempting to persist the same record results in an error from postgresql.
26
+
27
+ ![Alt text](/lib/aldrich/doc/operation_group.png?raw=true "Operation Group Handling")
28
+
29
+ ## Operation Group Translation
30
+
31
+ The Translation procedure for a group of Operation objects into actual ActiveRecord models is outlined here. It is a reasonably complicated process given that the dimentionality of our input data does not map 1-to-1 to our internal data structures. In addition, there is a great deal of domain knowledge dealing with business process, data policy, as well as association structure baked into each of the build procedures.
32
+
33
+ ![Alt text](/lib/aldrich/doc/operation_translation.png?raw=true "Operation Group Translation")
34
+
35
+ ## Model Build Procedure
36
+
37
+ The build procedure for each model instance is reasonably similar for each model. The logic that differs for each specific model has little to do with the overall procedure, and tends to only affect context or criteria for generating attributes or querying/detecting existing model instances in a given relation or table.
38
+
39
+ ![Alt text](/lib/aldrich/doc/model_build_procedure.png?raw=true "Model Build Procedure")
metadata ADDED
@@ -0,0 +1,111 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: aldrich
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.0
5
+ platform: ruby
6
+ authors:
7
+ - James Christie
8
+ - Christopher Hobbs
9
+ autorequire:
10
+ bindir: bin
11
+ cert_chain: []
12
+ date: 2016-05-20 00:00:00.000000000 Z
13
+ dependencies:
14
+ - !ruby/object:Gem::Dependency
15
+ name: rake
16
+ requirement: !ruby/object:Gem::Requirement
17
+ requirements:
18
+ - - "~>"
19
+ - !ruby/object:Gem::Version
20
+ version: '11'
21
+ type: :development
22
+ prerelease: false
23
+ version_requirements: !ruby/object:Gem::Requirement
24
+ requirements:
25
+ - - "~>"
26
+ - !ruby/object:Gem::Version
27
+ version: '11'
28
+ - !ruby/object:Gem::Dependency
29
+ name: pry
30
+ requirement: !ruby/object:Gem::Requirement
31
+ requirements:
32
+ - - "~>"
33
+ - !ruby/object:Gem::Version
34
+ version: '0.10'
35
+ type: :development
36
+ prerelease: false
37
+ version_requirements: !ruby/object:Gem::Requirement
38
+ requirements:
39
+ - - "~>"
40
+ - !ruby/object:Gem::Version
41
+ version: '0.10'
42
+ - !ruby/object:Gem::Dependency
43
+ name: rspec
44
+ requirement: !ruby/object:Gem::Requirement
45
+ requirements:
46
+ - - "~>"
47
+ - !ruby/object:Gem::Version
48
+ version: '3.4'
49
+ type: :development
50
+ prerelease: false
51
+ version_requirements: !ruby/object:Gem::Requirement
52
+ requirements:
53
+ - - "~>"
54
+ - !ruby/object:Gem::Version
55
+ version: '3.4'
56
+ - !ruby/object:Gem::Dependency
57
+ name: rails
58
+ requirement: !ruby/object:Gem::Requirement
59
+ requirements:
60
+ - - "~>"
61
+ - !ruby/object:Gem::Version
62
+ version: '4.2'
63
+ type: :development
64
+ prerelease: false
65
+ version_requirements: !ruby/object:Gem::Requirement
66
+ requirements:
67
+ - - "~>"
68
+ - !ruby/object:Gem::Version
69
+ version: '4.2'
70
+ description: A project that is an attempt to rehabilitate and generalize a data ingestion/importation
71
+ service originally built for a retail, e-commerce platform. It aims to provide a
72
+ robust solution for mapping intermediate representations of database records (CSV
73
+ at the moment) to persisted data. The process allows simulation, validation, and
74
+ control of the arrangement of the final data form. The ultimate goal is a project
75
+ that is flexible enough to use in a pure-Ruby, Rails, or standalone application
76
+ backed by data storage of your choice (as long as model representations can be accessed
77
+ via Ruby).
78
+ email:
79
+ - james.aaron.christie@gmail.com
80
+ - cmhobbs@member.fsf.org
81
+ executables: []
82
+ extensions: []
83
+ extra_rdoc_files:
84
+ - README.md
85
+ files:
86
+ - README.md
87
+ homepage: https://github.com/JamesChristie/
88
+ licenses:
89
+ - GPL3
90
+ metadata: {}
91
+ post_install_message:
92
+ rdoc_options: []
93
+ require_paths:
94
+ - lib
95
+ required_ruby_version: !ruby/object:Gem::Requirement
96
+ requirements:
97
+ - - ">="
98
+ - !ruby/object:Gem::Version
99
+ version: '0'
100
+ required_rubygems_version: !ruby/object:Gem::Requirement
101
+ requirements:
102
+ - - ">="
103
+ - !ruby/object:Gem::Version
104
+ version: '0'
105
+ requirements: []
106
+ rubyforge_project:
107
+ rubygems_version: 2.4.5.1
108
+ signing_key:
109
+ specification_version: 2
110
+ summary: A data ingestion, validation, simulation, and persistence tool
111
+ test_files: []