Real-Time ML
Unlike the traditional ML approach, where data scientists build the model using traditional data in an offline mode, real-time ML approaches use live data to continuously improve the model and make predictions.
With the high-speed growth in data in businesses, a good approach would be to improve model predictions based on new data as it comes in. this not only benefits the overall business processes but the customer satisfaction that comes with it.
Think of it with the example of google maps; based on your location at any given time, and you can get the right recommendations for restaurants or other facilities in that area or near you. Getting such a recommendation improves your experience not only on the applications but on the journey. This is as a result f real-time ML, where the newest data is used to make recommendations for you. Real-time machine learning approaches can also identify changing patterns in customer preferences without the need to retrain the models. This significantly reduces the time and cost that would have been incurred by the business.
Deploying a real-time machine learning model is commonly done using an event-driven architecture. In this case, the changing data is continuously fed to the model. This data pipeline modifies the model in real-time. This type of deployment requires very fast and low latency approaches ideally powered by in-memory technologies.
Online predictions- where predictions are made in real-time
This approach requires the lowest latency possible as customers require fast feedback. A slight delay in making the predictions would delay the customer. According to a study by Google, increasing the latency of webs searches reduces the overall daily number of searches per user (https://services.google.com/fh/files/blogs/google_delayexp.pdf). This implies that a solution is required to handle the online predictions. The first approach is through the use of batch predictions. This, however, works best if the input space is finite. In most cases, users are asked for categories to choose from. This works commonly in movie sites or music streaming. By choosing the categories, the batches that have already been predicted are consolidated and presented to the customer as recommendations.
Some of the areas where online predictions are important include high-frequency trading and fraud detection. To prevent fraud, banking systems need to identify the fraudulent patterns in real-time and block them.
Real-time prediction utilizes the dynamic features in data. But not all cases use dynamic features. Some of the predictions, especially batch predictions, use static features. Static features are slow-changing information such as age, gender and location. Dynamic features, on the other hand, are features based on the immediate environment or situation. As a business, understating the user’s needs or interests in the immediate moment enhances the chances that the recommendation made are relevant to them.
Online learning- where the system can take in new data and update the ML model.
Taking in new data and updating the ML model for predictions and recommendations is significantly an advantage for different companies. TikTok, for example, utilizes this approach to make its content highly relevant to the client. This also shows that recommendation systems are a perfect candidate for online learning. Online learning is preferred in the case of online content such as videos, articles, news and memes. YouTube has this concept well applied. When you watch a video on a given topic, most of the recommendations you get on what to watch next are relevant based on what you are watching. This change is seen in real-time. Changing the content immediately changes what is recommended to you. The recommendations also need to be relevant based on the context of the searches or the immediate data. Users need to get relevant recommendations or feedback from the data based on what it implies or refers to at that immediate juncture.
Since the online data keeps on shifting and the model training only sees each data point perhaps only once, there is no convergence. This shifts what lessons one has about model training. This is a challenge. Another challenge is on evaluation of the model. Since the data that the model uses is new and immediate, it does not make sense to evaluate it using old data. The new patterns learnt on the data would be a mismatch on the old data. Online learning is, however still, not fully practical.
Sources
https://hazelcast.com/glossary/real-time-machine-learning/
https://www.salesforce.com/eu/blog/2020/06/real-world-examples-of-machine-learning.html
https://huyenchip.com/2020/12/27/real-time-machine-learning.html
