One of the most important aspects of any modern eCommerce system is its ability to surface the most relevant products in response to the customers search queries.
Having a system that's robust, reliable and intelligent will go a long way in ensuring customer retention, boosting sales and attracting the envy of your competitors. Your online store could be selling thousands of great products at great prices. But what's the point of having all these amazing products, if your customers can't find the product they're looking for.
Having a sophisticated search engine that can intuitively understand what your users are looking for and adapt to their behavior over a period of time can bring massive advantages to your business over your competition.
Most modern eCommerce systems use advanced search engines like Elastic Search, Solr or Lucene etc., behind the scenes to power their product search experience. At Tech Fabric, we've got a lot of value out of using Azure Search and it's a standard fixture in many of applications we've built for our clients. We also occasionally use Elastic Search when more advanced capabilities were needed or when clients request an on-premise solution as opposed to purely cloud based service.
These systems not only boost the search capabilities of your store, they'll also play a big role in improving the overall performance and responsiveness of your application, as a lot of HTTP GET requests (search queries) can be offloaded to a specialized and a dedicated search service, while freeing up your system of record (Database) to handle only state change requests.
This pattern is also knows as CQRS Pattern - Command Query Responsibility Segregation. There will be separate Read and Write Models in your system. All read (HTTP GET) requests go to Azure Search and all state change (POST, PUT, DELETE) requests goes to Cosmos DB, since it's a system of record and ultimate source of truth.
The diagram below show a typical implementation of CQRS pattern we've used across many applications with very impressive results.
Cosmos DB has this amazing feature called 'Change Feed'. Put simply, it outputs the sorted list of documents that were changed in system and in the order in which there were modified. Any worker process can subscribe to those change and update down stream search indexes, data warehouses or any other temporary or persistent data store. The idea is to be able to create as many projections as need for any given system of record.
In the above example, we have a worker process running in the Service Fabric cluster that subscribes to all changes happening to a particular collection in Cosmos DB. This process will continuously iterate over the change feed, get the latest changes to the documents in the collection and processes it in real time. Any changes to the documents in Cosmos DB are instantly pushed to an index in Azure Search.
For ex, if a user purchases an item from your eCommerce store, the QOH (quantity on hand) for that product will be updated right away in Cosmos DB. The worker process will subscribe to the change feed in Cosmos DB, identifies the change in QOH and update the new value in Azure Search index, in near real-time. Next time, when a user goes to the product page, the change in quantity on hand is reflected on the interface.
This might be difficult to grasp at first or even feel weird, but once you get the hang of it and when implemented correctly, it can bring massive benefits to your business by breaking down data silos between different business units. By providing the most accurate information about your business to downstream applications in real-time, you're empowering them to make critical business decisions.
Armed with this knowledge, the business units that subscribe to this data can build rich applications that addresses their business needs, which otherwise would've been impossible without having real-time access to the data.
By segregating Read and Write Models, you're using the best of both the worlds. You get to leverage the power of search engines and their advanced search capabilities to show the most relevant data and at the same time, using the globally distributed and other cutting edge features of Cosmos DB to build robust highly scalable system of record.
In the next part, I'll walk you through the implementation process of setting up Cosmos DB collection, configuring change feed, consumers subscribing to change feed to update Azure Search index and finally demonstrating the use of separate Read and Write models. Stay tuned for more..
I hope you find this article helpful and I'd be happy if you share your own tips in the comments!