Monday, August 30, 2021

Data Science Study notes: Recommendation engine notes 2: collaborative filtering vs Content Base(Knowledge-Based)

Most recommender systems leverage two types of data:
1. Interaction Data, such as ratings, and browsing behaviors==> Collaborative Filtering method
2. Attribution Information, about each users and items ==> Content Base(Knowledge-Based) method

Hybrid Systems are then used to combined the advantages of these approaches to have a robust performing system across a wide variety of applications.

Intuitively, Collaborative Filtering Methods use the collaborative power of the ratings provided by multiple users to make recommendations and rely mostly on leveraging either inter-item correlations or inter-user interactions for the prediction process. It relies on an underlying notion that two users who rate items similarly are likely to have comparable preferences for other items.

Two types of methods from collaborative filtering:
Memory-based methods also referred to as neighborhood-based collaborative filtering algorithms, where ratings of user-item combinations are predicted based on their neighborhoods. These neighborhoods can be further defined as (1) User Based, (2) Item Based

Q: what is the basic principle that underlies the working of recommendation algorithms?
The basic principle of recommendations is that there are significant dependencies between user- and item-centric activity. For example, a user who is interested in hotels in New York City is more likely to be i nterested in other NYC hotels, rather than in Boston.

A recommender system, or a recommendation system, can be thought of as a subclass of information filtering system that seeks to predict the best “rating” or “preference”, that a user would give to an item, which is typically obtained by optimizing for objectives like total clicks, total revenue, and overall sales. For instance, which other car we should recommend for this user so that most likely they will submit a quote for that car!

The primary business goal of any recommender system is to provide users’ with a personalized experience! Either prediction problem(for other items) or Ranking problem(top k-item they like already or they might like).

LightFM is a Python implementation of several popular recommendation algorithms for both implicit and explicit feedback types. Importantly, it allows you to incorporate both item and user metadata into the traditional matrix factorization algorithms, making it possible to generalize to new items (via item features) and new users (via user features).

Here are some details for each of those 3 methods:
1. AWS Nepture is using the SQL/relationship way to get recommendation:
2. AWS EMR use the ALS algorithem to get recommendation(matrix factorization in Spark MLlib):
3. Use sagemaker for ML, DL aogorithem to get recommendation(deep matrix factorization, MXNET)


No comments:

Post a Comment

Data Science Study Notes: recommendation engine notes 1: Deep matrix factorization using Apache MXNet

Deep matrix factorization using Apache MXNet ( notes from Oreilly , github notebook ) Recommendation engines are widely used models th...