The more complex the e-commerce world becomes and the more systems are involved, the more important it is to have control over your data. Above all, that the data is accessible. The more systems you have, the less likely it is that your data is actually accessible. So how do you make it accessible?
The more complex the e-commerce world becomes and the more systems are involved, the more important it is to have control over your data. But almost even more important is that the data is accessible. The more systems you have, the less likely it is that your data is actually accessible, at least not in its entirety. I have lost count of how many times I have heard heads of e-commerce express a wish to be more nimble, to be able to quickly set up a new channel and test. But that it becomes too big a project, so they cannot manage it. Why is that?
The single biggest reason for that is that you do not have your data easily accessible for a new channel. When the new channel is to go up, it likely needs data from several different systems. You might have an e-commerce operation and want to set up screens in your stores, and the screens need at least product data but probably more than that. The e-commerce system or PIM system you have might not have the most modern APIs to get the data out, or you may have chosen not to bring all the data into the e-commerce system, and then suddenly the screens are supposed to get data from several places. And what felt like an easy and quick thing suddenly becomes something big.
It is easy to be fooled and think that if you just move to a composable architecture everything becomes so much easier, but often it becomes the exact opposite. In a composable architecture you often spread out data across different systems, because the different systems do a smaller number of things but do them really well. And the probability that one of the systems sits on all the data needed is fairly low.
It is also easy to be fooled and think that if you have a BI system or a data lake / warehouse system, then the problem is solved. But those systems are primarily designed for data collection and analysis, not for feeding other systems with data in different formats.
So what do you do? If you have your e-commerce operation and you want to put those screens up, it is tempting to build an integration against your e-commerce system so that the screens pull data from there. Maybe you are forced to push more data to the e-commerce system even though the data is only going to be passed on. But then you have only solved the problem here and now, and you end up in exactly the same spot when it is time for the next channel. The new channel will almost certainly have different data requirements than the last one.
To support setting up new channels, you need to gather all your data in one place. Which can feel counterintuitive in today's best-of-breed and composable-seeking world, to introduce a big monolith when it comes to data. But the downside of a monolith is when you mix data with logic, so that everything is tightly coupled. Here it is only about making data that exists in several other systems accessible so that it becomes much easier to consume.
Which system is best suited for that then? It is possible that you already have one or more systems that could be suitable. The most important requirements you need to set for such a system are:
1. Being able to handle many different types of data. You do not just want to send product data here, but all your data. Customers, orders, products, prices, categories, CMS content, all of it. The system you choose needs to have a flexible model to be able to handle that.
2. Search and filtering. Some channels may want to sync data periodically, while some channels will want to make real-time queries. To meet those requirements, the system needs to be able to handle search and filtering. If the system has support for GraphQL it is even better.
3. High performance and scalability. The system needs to have proven high performance to be able to support the channels that need to make real-time queries. A response of 300 milliseconds is not fast enough. You need to aim for response times well below 100 milliseconds. And the more channels you add, the more important it is that it is scalable and can handle the increased traffic.
4. Being able to handle data from many sources. You will want to send in data from several different systems here, and several sources should enrich the same entity. For example, your product data comes from your PIM, while product prices come from the e-commerce system or the ERP. The system has to be able to handle partial updates of the same product, customer and so on.
5. Being able to control content and sorting based on relevance. A use case here is of course listing products, and you want to sort them so that the most relevant product appears first. Or that a list of articles gives the most relevant article first.
There are many paths here. The cheapest path is to set up Elasticsearch yourself, which is an open-source search engine, and send data into it. Because it is a search engine, it handles searches and advanced filtering automatically. It is also very capable when it comes to different types of entities.
If you already have a standalone search engine for the e-commerce operation, that is also an option. Contact the company behind the search engine to get their view on it. There is also an interesting option here which is Occtoo. They are really at the forefront when it comes to making data accessible. We have no partnership with Occtoo and receive no kick-back for mentioning them, but it is always interesting when new players come along with fresh takes on an e-commerce operator's challenges.
Gathering all the data this way can of course become a large project in itself, but you have to start somewhere. A good path can be to not start with a real sales channel but with something simpler. Most e-commerce operators generate a wealth of different feeds to different systems, such as a Google Shopping feed, a Facebook feed, a Prisjakt feed and so on. A first step can be to gather enough data to generate these feeds. Once you have got there, you have enough product data in the system to, for example, be able to build a screen-in-store solution without much effort. If you then bring in customers and orders you can build an interface that customer service can use to look up customers. The more data you gather, the more use you get, but you do not have to push all data in at once.
Eventually you sit on enough data to be able to open new channels much faster than before. But at least as interesting is that you become less dependent on which systems you have in your landscape. If other systems are dependent on your data rather than on specific systems, it is easier for you to change which systems you have.
Not all problems are solved just because the data is accessible, of course. Completing a purchase, logging in, being notified when an item is out of stock, and so on, are all things a new channel may also need. But many of those things can come in later steps, and the most fundamental one is getting the data out.
If it is hard to get the data out, you will not even bother trying.
Related articles
For many years 'no code' and 'low code' have been pitched as the answer to the integration problem. The promise has been simple: no developers, no code, just draw a few arrows between systems and everything is solved. Read on to see why 'much code' is a better option.
For Swedish retailers there are plenty of ways to meet this threat by building on what the Chinese giants cannot offer: strong local brand stories, product safety and sustainability as competitive advantages, and a presence that ties digital and physical commerce together. Temu has no stores. Yet.
Price changes in e-commerce are often planned in advance and scheduled: either a change to the list price or a campaign price that takes effect on a specific date. Getting every system to show the new price at the right moment is a harder challenge than you might think, and in this article we walk through how to solve it with Nexus.
