shopify_transporter 2.0.0 → 2.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 51ea8531c355faaa542a5cac769df64be88ab1c7
4
- data.tar.gz: 8b46bab5dade9319600c9da397967f55aa7724ad
3
+ metadata.gz: 409e6ea11b360d21895a05e2f8f37fa7f924f7e4
4
+ data.tar.gz: 94c775ed84f0ecfbe3b8e7a5b084b0e990e5d4ff
5
5
  SHA512:
6
- metadata.gz: 60200c955aa3dd02d952835992f0b59e0cf1a8667daf45f7907ec346ed0b6df0b10cd3ff1b466d3a989c4ebf37bff1668a348f932e3308cbc6e31454107f0b87
7
- data.tar.gz: ff9d1c553e7617b9d84ceb0c933bbcfa3477a91e13e92986ef0e93f953ad69c030f5f92e5e5e90825f74e7b6fd672593a5b074fccb5873009f4b6e70a7a552b2
6
+ metadata.gz: 5eec50cdc6736cca34376105ecbfc8c3b954365a04c9224a112e64ace3f84ae98ccd320b5f09d75ecdc9a7d11e2c11e3384a1a53241c46e94c22f7341ed9a2f4
7
+ data.tar.gz: bef84c59176901f9c10a0353f81a22dbcd62dd1865b7e539ce0d3c6ffbe0c0c79e3376c620e08c502b2ab381681bdddcce2bf4ba03e60bf60479a5e129c2260f
data/README.md CHANGED
@@ -1,298 +1,41 @@
1
- # Transporter Tools
1
+ # Shopify Transporter
2
2
 
3
- The Transporter tool helps converts data from a third-party platform format into a format that can
4
- be imported into Shopify via the [Transporter app](https://help.shopify.com/manual/migrating-to-shopify/transporter-app).
3
+ `shopify_transporter` is a command-line tool that offers capabilities to extract and convert data from third-party platforms into a Shopify-friendly format.
5
4
 
6
- The conversions are currently limited to JSON structured data that has been exported from Magento 1.x via its
7
- SOAP API. More details about the exact structure of the JSON data, and additional scripts for procuding such
8
- data, will be made available in the near future.
5
+ This format can then be imported into Shopify via the [Transporter app](https://help.shopify.com/manual/migrating-to-shopify/transporter-app).
9
6
 
10
- Note: the Transporter app is available to Shopify Plus plans only.
7
+ `shopify_transporter` offers built-in support for migrations from Magento (version 1), as well as providing support for you to write your own transformations for other platforms.
8
+
9
+ *Note: the Transporter app is available to Shopify Plus plans only.*
10
+
11
+ ## Basic usage
12
+
13
+ Here's how you can get started migrating a store from a third-party platform onto Shopify using the Transporter suite of tools.
14
+
15
+ 1. Follow the [Getting started](https://github.com/Shopify/shopify_transporter/wiki/Getting-started) guide to install the necessary requirements.
16
+ 2. [Create a new project](https://github.com/Shopify/shopify_transporter/wiki/Create-a-new-project) for the source store.
17
+ 3. [Extract data](https://github.com/Shopify/shopify_transporter/wiki/Extract-data) from your source store.
18
+ 4. [Convert the extracted data](https://github.com/Shopify/shopify_transporter/wiki/Convert-data) into a Shopify-formatted CSV file.
19
+ 5. Use the [Transporter app](https://help.shopify.com/manual/migrating-to-shopify/transporter-app) to import the Shopify-formatted CSV file into a Shopify store
11
20
 
12
21
  ## Submitting Issues
13
22
 
14
23
  Please open an issue here if you encounter a specific bug with this library or if something is documented
15
24
  incorrectly.
16
25
 
17
- When filing an issue, please ensure that:
26
+ Before filing an issue, please ensure that:
18
27
 
19
28
  - The issue is not already covered in another open issue
20
29
  - The issue does not contain any confidential or personally identifiable information
21
30
  - The issue is specifically regarding the `shopify_transporter` gem and not related to the Transporter App.
22
31
 
23
-
24
- ## Installation
25
-
26
- ### Requirements:
27
- 1. A Ruby version of 2.4.0 or higher
28
- 2. You are able to install Ruby gems. Please visit [the Bundler website](https://bundler.io/) to troubleshoot issues with installing gems.
29
-
30
- We test and support the gem for Mac OS environments. While not officially supported, the gem may work on other operating systems provided they meet the above requirements.
31
-
32
- ### Installing the Transporter tool gem from rubygems:
33
-
34
- ```
35
- $ gem install shopify_transporter
36
- ```
37
-
38
- ### Running
39
- After you install the gem, you should find the executable `shopify_transporter` available in your path:
40
-
41
- ```
42
- $ which shopify_transporter
43
- /usr/local/bin/shopify_transporter
44
- ```
45
- ## Help and usage
46
-
47
- To view the usage and help for the `shopify_transporter` run the following command:
48
-
49
- ```
50
- $ shopify_transporter -h
51
-
52
- Commands:
53
- shopify_transporter convert --config=CONFIG --object=OBJECT file1.csv file2.csv ... # Converts your files into shopify formatted files.
54
- shopify_transporter generate STAGE_NAME --object=OBJECT # Generates a new pipeline stage for the specified object type
55
- shopify_transporter help [COMMAND] # Describe available commands or one specific command
56
- shopify_transporter new PROJECTNAME --platform=PLATFORM # Generates a new project structure for a platform
57
- ```
58
-
59
- ### Create a conversion project
60
-
61
- It's convenient to create a conversion project for each store that you want to migrate to Shopify.
62
- To create it, use the `new` sub-command:
63
-
64
- ```
65
- $ shopify_transporter new example-magento-conversion --platform=magento
66
- create example_magento_migration/Gemfile
67
- create example_magento_migration/config.yml
68
- create example_magento_migration/lib/magento/custom_pipeline_stages
69
- ```
70
-
71
- The `new` sub-command creates a project folder with the following:
72
-
73
- * a `Gemfile` that references the `shopify_transporter` gem. This Gemfile allows custom pipeline stages
74
- to refer to the base classes that are defined in the `shopify_transporter` gem.
75
-
76
- * a configuration file (`config.yml`) that provides the configuration required for each pipeline stage
77
- in the conversion process.
78
-
79
- * a folder to hold additional custom pipeline stages you may define later
80
-
81
- Switch to the project directory. For example:
82
-
83
- ```
84
- cd example_magento_migration
85
- ```
86
-
87
- As with any Ruby project, you need to run the following command before you create and run any custom
88
- pipeline stages:
89
-
90
- ```
91
- $ bundle install
92
- ```
93
-
94
-
95
- ### Convert records from the third-party platform to Shopify
96
-
97
- Run `shopify_transporter` and use the `convert` command to convert your objects from the
98
- third-party platform to the Shopify format. For example, the following command converts a
99
- JSON file that contains customers (*magento_customers.json*) from Magento to Shopify:
100
-
101
- ```
102
- shopify_transporter convert --config=config.yml --object=customer magento_customers.json > shopify_customers.csv
103
- ```
104
-
105
- In this example, the converted customer objects are saved to *shopify_customers.csv*. If errors occur during
106
- the conversion, then they appear in your terminal.
107
-
108
- ### Convert multiple files
109
-
110
- To convert multiple files to Shopify, separate the file names with a space:
111
-
112
- ```
113
- shopify_transporter convert --config=config.yml --object=customer magento_customers_1.json magento_customers_2.json ...
114
- ```
115
-
116
- ### Configuration file (_config.yml_)
117
-
118
- The configuraton file is generated when you create your conversion project. This file is specific to the
119
- third-party platform that you are converting to Shopify.
120
-
121
- Here's an example of a _config.yml_ file for converting customers from Magento:
122
-
123
- ```
124
- platform_type: Magento
125
- object_types:
126
- customer:
127
- record_key: email
128
- pipeline_stages:
129
- - TopLevelAttributes
130
- - AddressesAttribute
131
- - Metafields:
132
- type: all_platforms
133
- params:
134
- # Specify a custom namespace for your metafields with metafield_namespace.
135
- # Uses migrated_data by default.
136
- # metafield_namespace: migrated_data
137
- metafields:
138
- - website
139
- - group
140
- - free_trial_start_at
141
- ```
142
-
143
- ### record_key
144
-
145
- The `config.yml` file allows you to define the object type to convert. An object type needs a `record_key`,
146
- whose values must be unique among the other records in the file. For example, the default `record_key` for
147
- customers is the customer's email address.
148
-
149
- ```
150
- platform_type: Magento
151
- object_types:
152
- customer:
153
- record_key: email
154
- ...
155
- ```
156
-
157
- When you run `shopify_transporter` with the `convert` command, the input (third-party platform) files are
158
- read one-by-one and line-by-line. Each object in the input file must have a `record_key` value. Rows that
159
- have the same `record_key` value are considered to be part of the same object.
160
-
161
- ### Pipeline_stages
162
-
163
- The Transporter tool processes the input (third-party platform objects) through a series of pipeline
164
- stages. These stages, and the order in which they are to be processed, are defined in the `config.yml` file.
165
-
166
- In the example below, there are two pipeline stages for converting objects from Magento to
167
- Shopify: TopLevelAttributes and AddressAttributes.
168
-
169
- ```
170
- platform_type: Magento
171
- object_types:
172
- customer:
173
- record_key: email
174
- pipeline_stages:
175
- - TopLevelAttributes
176
- - AddressesAttribute
177
- ```
178
-
179
- Each pipeline stage's _convert_ method receives the row currently being processed, as well as the
180
- current state of the corresponding Shopify object being converted. The method's responsibility is to
181
- inject into the Shopify object the relevant attributes from the input row.
182
-
183
- The role of a pipeline stage is to examine the input rows and populate attributes on the Shopify object.
184
- For example, the `TopLevelAttributes` stage of a Magento customer migration looks for a column
185
- named `firstname` on the input, and then populates the Shopify object accordingly:
186
-
187
-
188
- ```
189
- record['first_name'] = input['firstname']
190
- ```
191
-
192
- Any changes that are made to the this record in a pipeline stage are permanent to the Shopify record
193
- associated with the `record_key`.
194
-
195
- The next pipeline stage that receives this record, receives the same input and the existing record which
196
- consist of:
197
-
198
- ```
199
- {
200
- 'first_name' => 'John',
201
- }
202
- ```
203
-
204
- Existing pipeline stages and the attributes are populated below.
205
-
206
- ### Magento v1.x customer
207
-
208
- ### AddressesAttribute
209
-
210
- Addresses are built from the `shipping_` and `billing_` prefixed fields. The Shopify object's `addresses`
211
- attribute is an array that consists of the `shipping_` prefixed attributes as the first address, and
212
- the `billing_` prefixed attributes as the second.
213
-
214
- ```
215
- {
216
- 'addresses': [
217
- {
218
- 'first_name': ...,
219
- 'last_name': ...,
220
- 'address1': ...,
221
- 'address2': ...,
222
- 'city': ...,
223
- 'province': ...,
224
- 'country_code': ...,
225
- 'zip': ...,
226
- 'company': ...,
227
- 'phone': ...,
228
- },
229
- ]
230
- }
231
- ```
232
-
233
- ### Metafields
234
-
235
- See the metafields section in the All Platforms section below.
236
-
237
- ### All Platforms
238
-
239
- Some stages are a little more versatile and can support input rows from arbitrary third-party platforms. These stages
240
- are usually extensible through defining parameters in the `config.yml`.
241
-
242
- #### Metafields
243
-
244
- The `convert` command converts the most popular metafields or custom fields from the third-party platforms into
245
- Shopify [metafields](https://help.shopify.com/manual/products/metafields). You can view the default metafields and
246
- add others in your `config.yml` file.
247
-
248
- ## Adding customized stages
249
-
250
- If the Shopify object that is generated is missing attributes or if a pipeline stage fails, it could be because there are
251
- unexpected headers defined in the third-party CSV data.
252
-
253
- To define your own customized stages, run the following command:
254
-
255
- `shopify_transporter generate YourCustomStage --object customer`
256
-
257
- A file named `your_custom_stage.rb` is added to the `lib/magento/custom_pipeline_stages` directory. You can modify this file to add your custom conversion logic.
258
-
259
- ## Limitations
260
-
261
- The `convert` command currently only converts customer and order JSON objects that have been exported by SOAP API
262
- from Magento 1.x.
263
-
264
32
  ## Contributing
265
33
 
266
- ### Running unit tests
267
-
268
- We use rspec to run our test suite:
269
- `bundle exec rspec`
270
-
271
- ### Running the linter
272
-
273
- It's important that all of the code introduced passes linting. To run it manually:
274
- `bundle exec rake rubocop`
275
- To automatically resolve any basic linting issues:
276
- `bundle exec rake rubocop:autocorrect`
277
-
278
- ### Building and installing locally
279
- To build locally run the following command:
280
-
281
- ```
282
- $ bundle exec rake build
34
+ If you'd love to see a particular feature as part of `shopify_transporter`, submit a pull request adding that feature!
283
35
 
284
- shopify_transporter 1.0.0 built to pkg/shopify_transporter-1.0.0.gem.
285
- ```
36
+ Please read our [contribution guidelines](https://github.com/Shopify/shopify_transporter/wiki/Contributing) before submitting a pull request.
286
37
 
287
- Then, you can install the gem system-wide by running:
38
+ ## License
288
39
 
289
- ```
290
- $ gem install pkg/shopify_transporter-1.0.0.gem
40
+ `shopify_transporter` is listed under the MIT license. See [LICENSE.md](https://github.com/Shopify/shopify_transporter/blob/master/LICENSE) for more details.
291
41
 
292
- Successfully installed shopify_transporter-1.0.0
293
- Parsing documentation for shopify_transporter-1.0.0
294
- Installing ri documentation for shopify_transporter-1.0.0
295
- Done installing documentation for shopify_transporter after 0 seconds
296
- 1 gem installed
297
- ```
298
- Your locally built gem is now installed system-wide.
@@ -52,6 +52,8 @@ module ShopifyTransporter
52
52
  exporter.run
53
53
  rescue Exporters::ExportError => error
54
54
  extract_error(error.message)
55
+ rescue SystemExit, Interrupt
56
+ $stderr.puts "\n...exiting"
55
57
  end
56
58
 
57
59
  no_commands do
@@ -45,7 +45,7 @@ module ShopifyTransporter
45
45
  attributes = map_from_key_to_val(COLUMN_MAPPING, input)
46
46
  attributes['published'] = published?(input)
47
47
  attributes['published_scope'] = published?(input) ? 'global' : ''
48
- attributes['published_at'] = published?(input) ? input['updated_at'] : ''
48
+ attributes['published_at'] = input['updated_at'] if published?(input)
49
49
  append_images_to_current_record!(input) if input['images'].present?
50
50
  attributes['tags'] = product_tags(input) if input['tags'].present?
51
51
  attributes['options'] = product_options(input) if input['option1_name'].present?
@@ -1,4 +1,4 @@
1
1
  # frozen_string_literal: true
2
2
  module ShopifyTransporter
3
- VERSION = '2.0.0'
3
+ VERSION = '2.0.1'
4
4
  end
@@ -43,6 +43,14 @@ class TransporterTool
43
43
  end
44
44
  end
45
45
 
46
+ class UnexpectedJSONStructureError < ConversionError
47
+ def initialize
48
+ super(
49
+ 'Unexpected JSON structure detected. The expected format is an array of JSON objects: [{}, {}, ..., {}]'
50
+ )
51
+ end
52
+ end
53
+
46
54
  def initialize(*files, config, object_type)
47
55
  config_file(config)
48
56
  input_files(*files)
@@ -113,8 +121,9 @@ class TransporterTool
113
121
  file_data = File.read(file_name)
114
122
  parsed_file_data = Yajl::Parser.parse(file_data)
115
123
  return if parsed_file_data.nil? || parsed_file_data.empty?
116
-
124
+ raise UnexpectedJSONStructureError unless expected_json_structure?(parsed_file_data)
117
125
  record = 1
126
+
118
127
  parsed_file_data.each do |json_row|
119
128
  process(json_row, file_name, record)
120
129
  record += 1
@@ -252,4 +261,8 @@ class TransporterTool
252
261
  def key_required_from_config
253
262
  @config['object_types'][@object_type]['key_required']
254
263
  end
264
+
265
+ def expected_json_structure?(input)
266
+ input.is_a?(Array) && input.all? { |json_row| json_row.is_a?(Hash) }
267
+ end
255
268
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: shopify_transporter
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.0.0
4
+ version: 2.0.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Shopify
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2018-10-02 00:00:00.000000000 Z
11
+ date: 2018-10-04 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: bundler