Interaction

Shopify

Last Authored: Monday, July 24th, 2023

This Interaction service allows you to pull data from your Shopify online store.

Requirements

For Shopify Interaction layer to work we require an Admin API Access Token. Please read creating a custom app for more info.

Steps on retreiving Admin API Access Token

  1. Go to your Online store and click Settings > Apps and Sales Channels > Develop apps

  2. Click on the button that says Create an app

  3. This should open up a modal asking for an App name

  4. Give the app a name and click Create app

  5. Once the app has been created, click on Configure Admin API Scopes

  6. This should open up a page which allows you to enable scopes of the Admin API. Currently we only offer two integrations in the Interaction layer (Products and Inventory). For these two to work you need to enable the read_products, read_inventory and read_locations scope.

  7. Click the Save button, which should change into Install app button

  8. Click the Install app button which opens up a modal and click the Install button in the modal

  9. You should now have the ability to see the Admin API token. The Reveal token once CTA will display the token.

Shopify Products

This services requires read_products scope on the custom app.

This Interaction layer allows you to fetch your products data. It utilizes the Products > Product resource on Shopify's Admin API.

The UI for this layer will ask for the following inputs:

  • Interaction Name - Used for dashboarding and internal resource naming

  • Shopify API Version - the API version of Shopify you would like to use. This can be retrieved from Shopify's releases page

  • Shopify Admin API Access Token - API Token created when creating the custom app

  • Frequency - How often do you want to pull the data. Currently we only support three options, hourly, daily and monthly.

Behind the scenes

To make this data fetch work, we deploy three AWS resources:

  1. Lambda

  2. SQS - Simple Queue Service

  3. Events Rule

One Lambda resource is deployed, written in Node, which is responsible for three tasks. First, it hits Shopify's Product API and pulls data in JSON format. Second, it extrapolates this data and map it to products's variants so that all values being read are unique. This data is then sent to an SQS queue which is used as the reading source in the extraction stage. When this service is deployed, we set limits to how many of these Lambdas can run in parallel. By default, AWS sets it to 1000 however Shopifys Admin api is limited to 2 calls/sec or 4 calls/sec if you have a Shopify Plus store. Hence, to be safe we have configured parallel execution of this function to 2 i.e only 2 functions can run at a time.

Two SQS resources are deployed, 1 responsible to fetch the next batch of products and the other to store the extrapolated data. Shopify's Product API is limited to fetching a max of 250 products in one call. However in Project Compound, we limit the fetch to 10 products to optimize cost and performance of the node function. Hence, to grab all the products we send a header value to the next batch SQS queue which queues the Lambda function to run with that header value to fetch the next batch of products.The other SQS queue is to hold the extrapolated data which is utilized in the Extraction layer.

One Event Rule is deployed which triggers the Lambda resource based on the frequency option selected.

Compound workflow

Shopify Inventory

This services requires read_locations and read_inventory scope on the custom app.

This Interaction layer allows you to fetch Locations and Inventory data. It utilizes the Inventory > Inventory Level and Inventory > Locations resource on Shopify's Admin API.

The UI for this layer will ask for the following inputs:

  • Interaction Name - Used for dashboarding and internal resource naming

  • Shopify API Version - the API version of Shopify you would like to use. This can be retrieved from Shopify's releases page

  • Shopify Admin API Access Token - API Token created when creating the custom app

  • Frequency - How often do you want to pull the data. Currently we only support three options, hourly, daily and monthly.

Behind the scenes

To make this data fetch work, we deploy three AWS resources:

  1. Lambda

  2. SQS - Simple Queue Service

  3. Events Rule

Two Lambda resources are deployed, written in Node, each is responsible for three tasks. First Lambda, hits Shopify's Inventory > Locations API and pulls data in JSON format. This data is then sent to an SQS queue. The second lambda reads the data from the the queue and fetches Inventory > Inventory Levels data. This data is extrapolated and sent to another SQS queue for the extraction layer to use. The Inventory Levels data also contains a pagination header which is sent to a separate SQS queue. The lambda responsible to fetch Inventory Levels data is also attached to this paginated queue and gets triggered to fetch the next batch of Inventory levels.

When this service is deployed, we set limits to how many of these Lambdas can run in parallel. By default, AWS sets it to 1000 however Shopifys Admin api is limited to 2 calls/sec or 4 calls/sec if you have a Shopify Plus store. Hence, to be safe we have configured parallel execution of this function to 2 i.e only 2 functions can run at a time.

Three SQS resources are deployed, 1 responsible to store the locations data, another to store the extrapolated Inventory Levels data and the other to store the pagination header data.

One Event Rule is deployed which triggers the Lambda resource based on the frequency option selected.

Shopify Products flow