Introduction to Recommendation System

A recommendation system is a branch of machine learning focused on building recommendation models that predict user preferences and generate personalized item suggestions to users by analyzing their past interactions, behaviour, item attributes, and user demographic data. These systems are widely used across various domains, such as e-commerce, media streaming, and social networking platforms, where they help users discover relevant products, movies, books, or other content as per their previous interactions. Through personalized item suggestions, recommender systems will improve user satisfaction, boost engagement, increase user retention, and sales for businesses by presenting the most relevant options specifically for each user.

Sample Business Applications

Recommendation models are actively integrated into e-commerce platforms to solve multiple usecases ,which drives the business value.

1. Sequential Recommendations in E-commerce Platforms:

Sequential Recommendation can be applied in e-commerce platforms to suggest products based on users’ browsing and purchase history. By analyzing the sequence of products viewed or bought by users, businesses can provide personalized recommendations, thereby increasing sales and customer satisfaction.

Scenario: In e-commerce platforms, analyzing the sequential patterns of user interactions can reveal insights into user preferences and purchasing habits. For example, if a user purchases running shoes followed by sports apparel, the platform can identify this sequence. Based on this information, the platform can recommend complementary products such as running socks, water bottles, or fitness trackers. These personalized recommendations not only increase the likelihood of additional purchases, but also enhance the overall shopping experience for the user.

2. Personalized Product Suggestions:

Personalized recommendations in e-commerce platforms involve suggesting products to users based on their browsing history, purchase behavior, and preferences. By analyzing user interactions and historical data, e-commerce platforms can deliver tailored product recommendations, thereby enhancing user experience and increasing sales.

Scenario: Consider an established e-commerce platform where users browse a vast array of products across different categories. By analyzing user behavior, such as past purchases, items added to the cart, and products viewed, the platform can generate personalized product suggestions. For example, if a user frequently purchases electronics and gadgets, the platform can identify this preference. Based on this insight, the platform can recommend similar products, accessories, or related items that align with the user’s interests. These tailored recommendations not only improve user satisfaction but also increase the likelihood of conversion and repeat purchases.

3. Consumer Goods and FMCG (Fast-Moving Consumer Goods):

In industries such as consumer goods and FMCG, where products are purchased frequently and regularly, optimizing repeat purchases is essential for businesses. The challenge lies in effectively recommending staple items or commonly purchased products to customers based on their purchase history and consumption patterns.

Scenario: For example, in the food and beverage sector, a grocery delivery service can use repeat purchase recommendation algorithms to suggest staple items or commonly purchased groceries based on a customer’s purchase history. By analyzing past orders and consumption patterns, the platform can anticipate when certain items are likely to be depleted and recommend replenishment to streamline the ordering process for customers.

4. Retail Inventory Management:

By predicting the next occurrence of product purchases, retailers can optimize their inventory levels and ensure sufficient stock availability. This enables them to prevent stock outs, minimize overstocking, and enhance overall supply chain efficiency.

Scenario: In e-commerce platforms or even retail vendors, offer subscription services with various renewal intervals, such as monthly or bimonths. Additionally, by analyzing recurring purchase patterns, retail businesses can better manage inventory, minimizing stock outs or overstocking. By optimizing inventory levels, businesses can ensure stock availability, consistently serving users’ needs.

5. Subscription Services:

Subscription-based businesses, such as subscription boxes, meal kit delivery services, or subscription-based software platforms, rely on maintaining a steady stream of recurring revenue through subscription renewals. However, managing subscription renewals for plans that require manual reminders, especially when renewal interval patterns are complex, presents a challenge.

Scenario: For example, in the case of a newspaper subscription service, some customers may only renew their subscriptions during specific periods, such visits to their vacation homes. In such cases, repeat purchase recommendation algorithms can play a crucial role in identifying subscription renewal dates and providing timely reminders to users. By analyzing subscribers’ renewal history and understanding complex renewal patterns, the recommendation system can enhance retention rates and ensure a consistent flow of revenue for the business.

Similar to e-commerce platforms, recommendation algorithms can also be applied in various industries to enhance user experience, engagement, and ultimately drive business value. Few examples are:

• Entertainment and Media: Content streaming platforms such as movies, music, and articles based on past consumption.
• Education: Generate course recommendations and customize learning paths based on the student’s interests and learning style.
• Social Media: Content suggestions, friends, and groups suggestions based on user interactions and interests.
• Healthcare: Medication reminders, health care plans suggestions based on patient medical history and health data. These are just a few examples; many other industries are actively exploring ways to integrate recommendation models to drive value and enhance business outcomes.

Recommendation Model Evaluation

These evaluation metrics collectively provide insights into different aspects of the performance of a recommendation model, including the diversity, quality, predictive accuracy, and overall correctness of the item suggestions/recommendations the model generates.

Let’s explore and interpret the metrics used in evaluating the recommendation models:

a. Coverage

What it tells you: Coverage measures the proportion of all possible items that the system can recommend.

Intuition:

• High coverage means the system can recommend a wide variety of items, making it more diverse and less biased toward popular items.
• Low coverage suggests that the recommendations are limited to a smaller set of items, which may not cater to all users.

Example Inference: If coverage is 90%, the system can recommend 90% of the items in the catalog, ensuring diverse options for users.

b. NDCG at K (Normalized Discounted Cumulative Gain at K)

What it tells you: NDCG measures how well the recommendations are ranked, focusing on the position of relevant items within the top K recommendations.

Intuition:

• High NDCG means relevant items appear near the top of the list, making the recommendations more useful to users.
• Low NDCG indicates that relevant items might be buried deeper in the list, reducing user satisfaction.

Example Inference: If NDCG@10 is high, users are more likely to see relevant recommendations within their top 10 suggestions, enhancing their experience.

c. Accuracy

What it tells you: Accuracy is the percentage of correct predictions (relevant vs. irrelevant recommendations) made by the system.

Intuition:

• High accuracy means the system is generally good at distinguishing relevant recommendations from irrelevant ones.
• However, it doesn’t capture how well the system handles edge cases, like rare items or user-specific preferences.

Example Inference: If accuracy is 95%, most recommendations are likely correct, but it doesn’t ensure that all relevant items are included.

d. ROC AUC (Receiver Operating Characteristic Area Under the Curve)

What it tells you: ROC AUC evaluates how well the system can separate relevant from irrelevant items across different thresholds.

Intuition:

• High AUC means the system can effectively rank relevant items higher than irrelevant ones, regardless of the cut-off threshold.
• Low AUC suggests that the system struggles to distinguish relevant items from irrelevant ones. Example Inference: If AUC is 0.95, the system is very reliable at ranking relevant items consistently across all users.

e. Precision

What it tells you: Precision measures the proportion of recommended items that are actually relevant.

Intuition:

• High precision means most of the recommendations made are useful and relevant to the user.
• Low precision indicates that users are being shown many irrelevant items, which could reduce trust in the system.

Example Inference: If precision is 80%, 8 out of every 10 recommendations are likely to be relevant.

f. Recall

What it tells you: Recall measures the proportion of all relevant items that are successfully recommended.

Intuition:

• High recall means the system is good at identifying all the relevant items, even if it occasionally includes irrelevant ones.
• Low recall suggests that many relevant items are being missed.

Example Inference: If recall is 70%, the system is finding 70% of all the relevant items for a user, but some are still being left out.

g. Recurrence Rate

Definition: The proportion or frequency of recurrent events or patterns within a dataset.

What it tells you intuitively:

• A higher recurrence rate means the dataset contains frequent repeated events or patterns.
• A lower recurrence rate suggests fewer or rare repetitions in the data.

Example Inference: If the recurrence rate is 60%, it means 60% of the events in the dataset are part of some repeating pattern. This helps identify how cyclical or predictable the data is.

h. Mean Absolute Error (MAE)

Definition: The average absolute difference in days between the predicted and actual recurrence.

What it tells you intuitively: MAE shows how far off the predictions are, on average, from the true recurrence intervals, ignoring whether the difference is an overestimation or underestimation. Lower MAE means more accurate predictions.

Example Inference: If MAE is 3 days, the predicted recurrence intervals are, on average, 3 days away from the actual intervals. This gives a straightforward sense of how precise the predictions are.

i. Mean Squared Error (MSE)

Definition: The average squared difference in days between predicted and actual recurrence.

What it tells you intuitively:

• MSE penalizes larger errors more heavily because it squares the differences.
• Lower MSE means the model is generally accurate and avoids large mistakes in predictions.

Example Inference: If MSE is 16 days², it indicates that while most predictions may be close, any larger errors in the recurrence interval predictions significantly impact the score. This makes it useful for identifying models prone to big errors.

Evaluation metrics for specific algorithms

Recommendation models vary in type, as previously mentioned. Hence, there is no single generic approach to evaluate the models. Instead, they can be evaluated using different metrics specific to each algorithm.

The evaluation metrics for information retrieval algorithms are as follows:

a. Information Retrieval

Recommendation type	Algorithms	Evaluation Metrics
Sequential Recommendation	SubSeq	Coverage: Coverage measures the proportion of all possible items that the system can recommend. NDCG at K (Normalized Discounted Cumulative Gain at K): NDCG measures how well the recommendations are ranked, focusing on the position of relevant items within the top K recommendations. Accuracy: Accuracy is the percentage of correct predictions (relevant vs. irrelevant recommendations) made by the system.
Personalized Recommendation	Light FM	ROC AUC (Receiver Operating Characteristic Area Under the Curve): : ROC AUC evaluates how well the system can separate relevant from irrelevant items across different thresholds. Precision: : Precision measures the proportion of recommended items that are actually relevant. Recall: Recall measures the proportion of all relevant items that are successfully recommended.
	Pixie	Precision: Precision measures the proportion of recommended items that are actually relevant. Recall: Recall measures the proportion of all relevant items that are successfully recommended. Coverage: Coverage measures the proportion of all possible items that the system can recommend.
Recurring Purchase Recommendations	Recurrence Finder	Recurrence Rate: The proportion or frequency of recurrent events or patterns within a dataset. Mean Absolute Error (MAE): The average absolute difference in days between predicted and actual recurrence. Mean Squared Error (MSE): The average squared difference in days between predicted and actual recurrence.

Each of these evaluation metrics serves a specific purpose in assessing the performance of a respective model, providing insights into its performance and reliability.