According to a recent study published by The Business Research Company, “The global same-day delivery services market grew from $6.44 billion in 2022 to $7.93 billion in 2023 at a compound annual growth rate (CAGR) of 23.1%.” To keep up with this demand for fast delivery, eCommerce retailers increasingly require an efficient and robust Order Management System (OMS) to implement omnichannel strategies like Buy Online Pick Up In-Store (BOPIS), Ship from Store, Ship to Store, and Buy Online Return In-Store. An OMS plays a crucial role in implementing these strategies by routing eCommerce orders to the most efficient fulfillment location, thereby allowing retailers to sell more inventory and deliver online orders to their customers quickly and cost-effectively.
To effectively route orders, these retailers must ensure that all product, inventory, and order data from their eCommerce platform is downloaded and synchronized with their OMS. Furthermore, seamless OMS integration is required with the retailer’s Shopify eComm, ERP, and POS systems. And while many software providers can import data from Shopify when order volumes are low, upscaling can pose challenges such as incomplete data downloads and slow download speeds due to the restrictions of Shopify’s APIs, hindering seamless order routing and data synchronization with their OMS.
With the input of our enterprise-level Shopify clients’ firsthand experience with the challenges of Shopify-OMS integration, let’s take a deep dive into 4 essential considerations you will need to know as you plan your omnichannel optimization strategy for providing top-notch service and same-day, next-day, and two-day delivery to your customers.
1. Optimize for Leaky Bucket Algorithm
Shopify, like many other eCommerce platforms, uses the leaky bucket algorithm to manage the flow of API requests from third-party apps and integrations. According to this algorithm, third-party apps can place as many API requests as they want but Shopify will always respond at a fixed rate.
To better understand this concept, let's imagine a bucket with a capacity of 60 marbles, representing the access granted to each third-party app. When an app sends an API request, it must add a marble to the bucket. As Shopify returns API responses, a marble is removed from the bucket for each response, ensuring that there is always room for more requests. However, if the bucket becomes full, Shopify will send an error message, and the apps will need to wait until there is enough space in the bucket to continue sending requests.
Fig.1(i): API requests when not exceeding the bucket size
Fig.1(ii): API requests when exceeding the bucket size
The leaky bucket algorithm plays a crucial role for Shopify as it safeguards the API from being overloaded, ensuring optimal performance. By regulating the rate at which API requests are processed, this algorithm prevents Shopify's servers from becoming overwhelmed and helps maintain a consistent level of performance for all users. It acts as a protective measure to ensure the smooth operation of the platform and deliver a reliable experience to merchants and customers alike.
Fig.2: Shopify's API limit
Recommendation: Order Management Systems should intelligently download data like customer orders from Shopify in a way that it can get optimal throughput from Shopify's APIs while avoiding disruptions caused by throttling.
2. Downloading and Syncing Products From Shopify
For the Order Management System (OMS) to accurately store inventory counts, capture orders, and route them, it needs to download product information from Shopify. However, if the products are not downloaded and updated correctly in the OMS, errors can arise during the synchronization of product inventory.
Shopify has a limit of 250 product styles that can be downloaded in a single API call. Each product style can have multiple variants, and in some cases, these product styles can have a large number of variants. Attempting to download 250 product styles, each with numerous variants significantly increases the size of the response file provided by Shopify. If the response file surpasses Shopify's maximum file size limit, an error occurs and the product download process fails. This limitation can pose challenges when trying to retrieve and synchronize comprehensive product data from Shopify to the OMS.
Recommendation: Download up to 100 products in an API call. While this approach may require additional API calls to retrieve all the desired products, it enables a more reliable and comprehensive download process.
3. Syncing inventory to Shopify
Maintaining accurate inventory counts is crucial for preventing order cancellations, minimizing markdowns, and maximizing the return on inventory investment. Accurate inventory information ensures that all orders are captured and fulfilled when inventory is available at any given location. The Order Management System (OMS) plays a pivotal role as the central hub for inventory availability, OMS integration with various systems is necessary to fetch accurate inventory data for eCommerce operations.
For Shopify retailers, it is essential to sync inventory counts between the OMS and all fulfillment locations daily, preferably in the morning. This synchronization ensures that the OMS has the most up-to-date information regarding available inventory. Additionally, the OMS should facilitate frequent inventory syncing throughout the day to maintain near real-time updates of inventory count changes at fulfillment locations.
When syncing inventory, retailers should take the following considerations into account:
-
Bulk inventory sync: Every morning, the OMS should receive an inventory feed from the ERP (which serves as the master of inventory), and the inventory feed should contain all of the products’ refreshed Quantity on Hand (QOH) inventory numbers at various locations. Using this refreshed inventory number, OMS calculates and pushes sellable inventory to Shopify eComm.
Fig.3: Inventory Flow
Managing inventory for enterprise retailers with thousands of products and numerous variants poses a significant challenge. When attempting to edit the inventory of such a vast number of products simultaneously, it results in a high volume of API requests to Shopify that surpass its processing capacity. Consequently, Shopify is unable to process these API requests, leading to inaccurate inventory counts. This inaccuracy increases the risk of overselling, where more products are sold than available, or underselling, where available inventory is not accurately reflected.
Recommendation: To achieve inventory accuracy while minimizing API calls, it is recommended that the Order Management System (OMS) reads the inventory feed from the ERP system. The OMS should compare the inventory counts between ERP and OMS, adjusting the differences accordingly. After syncing the inventory, the OMS can recalculate the Available to Promise (ATP) inventory and compare it with Shopify. Only the delta, representing the discrepancy in inventory counts, needs to be shared with Shopify for updating specific products. This approach ensures inventory accuracy with fewer API calls by selectively updating inventory for products that require adjustments, resulting in efficient inventory management for Shopify retailers.
Fig.4 : Inventory adjustments from ERP to OMS
Fig.5: Inventory adjustments from OMS to Shopify
-
Near real-time inventory updates: Syncing inventory once a day for Shopify retailers can result in discrepancies between available inventory and actual stock levels. In-store customer purchases and ongoing inventory replenishment lead to constant changes in inventory count throughout the day. Failing to record these changes in real-time can result in inaccurate inventory availability for online customers, potentially causing issues such as overselling or underselling. To address this, it is recommended that Shopify retailers implement real-time inventory adjustments to accurately reflect in-store sales and inventory updates. By promptly recording and synchronizing these changes, retailers can ensure that online inventory availability is always up-to-date, reducing the risk of inventory inconsistencies and improving the overall customer experience.
Recommendation: Adjust Shopify inventory throughout the day to reflect in-store sales and receiving. Record and push updates in batches at intervals: 30 mins for high throughput, and 1 hour for low throughput.
4. Downloading Orders from Shopify
The primary role of an Order Management System (OMS) is to efficiently route orders to the most suitable fulfillment location. Smart order routing can help reduce the cost of delivery, improve inventory utilization, and ensure speedy delivery to customers. To route orders effectively, the OMS needs to download them from the Shopify platform. Once the order routing is completed and the fulfillment process has commenced, Shopify retailers require the fulfillment status of the orders to update them on Shopify and communicate the information to customers. However, there are several challenges that Shopify retailers commonly encounter while downloading eCommerce orders from Shopify.
-
Exceeding Shopify’s response file size limit while downloading orders from Shopify: Shopify allows downloading a maximum of 250 orders in one API call. It is important to note that each order may contain multiple line items, which can significantly increase Shopify's response file size. If the response file size exceeds Shopify's maximum limit, an error will be returned, and the orders will not be downloaded into the OMS.
Recommendation: Download only 100 orders in an API call. While this does not utilize the full capacity of the API, it reduces the possibility of a missed download.
-
Missing order downloads due to the time gap between two subsequent jobs: The OMS typically utilizes scheduled jobs to carry out its tasks at predefined intervals. Among these tasks are jobs specifically designed to download orders created within specific time durations in Shopify. However, there is a potential for missing orders if they are placed within the narrow time gap between two consecutive jobs. This means that orders falling within these microsecond time gaps may not be captured during the scheduled downloads.
For example, a job downloads orders between 01:00:00 PM to 01:15:00 PM and a second job downloads orders between 01:15:00 PM to 01:30:00 PM. In this scenario, if an order is created in Shopify at 01:14:140 PM but is added to Shopify’s database at 01:15:01 PM, the order will not be downloaded by either of the jobs.
Here, if an order is created in Shopify at 01:14:140 PM but is added to Shopify’s database at 01:15:01 PM, the order will not be downloaded by either of the jobs.
Recommendation: To ensure that no orders are missed, add a buffer time. For example, if the last job downloaded orders from 01:00:00 PM to 01:15:00, then the subsequent job will download orders from 01:14:00 to 01:30:00 PM.
Fig.6: Buffer time to ensure no orders are missed with time overlap between two jobs
-
Failing to download an order due to a missing or invalid shipping address: Two scenarios come into play here:
Case 1: In certain cases, customer service representatives (CSRs) may create orders for customers through the Shopify admin interface. However, these orders may be initially created without a shipping address. Subsequently, the CSR teams may reach out to the customers to obtain the necessary address information and add it to the order. It's worth noting that certain Order Management Systems (OMSs) are set up to exclude orders that lack an address or have an invalid shipping address during the download process. Consequently, if a CSR fails to add the address to an order, it will not be downloaded and will remain unfulfilled within the OMS.
Case 2: If an OMS is configured for a specific country and lacks location data for other countries in its database, it may encounter difficulties when processing orders from unsupported countries. When a customer places an order with an address that is not mapped in the OMS, Shopify will accept the order, but the OMS will perceive the address as invalid and subsequently fail to download the order. For instance, let's consider a retailer that has set up their OMS in the United States but operates in multiple countries, including Australia. If a customer from Australia visits the retailer's US Shopify eCommerce website and places an order with an Australian address, the OMS will be unable to map the customer's location and will not download the order due to the perceived invalid delivery address.
Recommendation: The OMS should download the orders even if the customer's delivery address is missing or invalid. It should then create a note in Shopify to update the address later. At their convenience, CSR teams can contact customers and edit/add customers’ delivery addresses. However, if not downloaded, the orders will not be brokered and fulfilled. The order will either be delayed or canceled, which may lead to a negative customer experience.
-
Failing to download an order due to missing product: It is possible that a product in Shopify is not created in OMS yet. All the orders placed for these products will not be downloaded in the OMS since it will not be able to associate these orders with any products.
Recommendation: To download such orders, OMS can create a dummy product and download the order. Then, the original product ID can be updated or swapped with a similar product.
How HotWax Commerce downloaded ~50K orders in a day for an enterprise client?
By implementing the recommendations above to simplify the Shopify-OMS integration process!
At HotWax Commerce, we have successfully integrated HotWax Commerce with Shopify eCommerce and POS systems to seamlessly share customer, inventory, and product data across multiple platforms. Throughout this process, we encountered several obstacles and refined our OMS integration to handle peak season demands and download and process up to fifty thousand eCommerce orders in a day. This approach effectively mitigated any potential data leaks during high-order traffic periods.
The recommendations outlined in this process were tailored to cater to the needs of retailers with thousands of SKUs, a high volume of orders, and frequent inventory updates in Shopify. However, these specifications can be adjusted to fit other Shopify retailers' profiles.
* * *
HotWax Commerce is a cloud-based Omnichannel Order Management solution that has ready OMS integration with Shopify eComm and Shopify POS. With HotWax Commerce OMS, Shopify retailers will be able to ensure inventory accuracy, optimally route orders to fulfillment centers, successfully fulfill online orders from stores, and also keep track of fulfillment activities. Should you encounter any difficulties in Shopify-OMS integration, feel free to reach out to us for expert advice.
