Session 5: Diversity, Novelty and Serendipity

Date: Wednesday, Oct 8, 10:30-13:00
Moderator: Dietmar Jannach

• Improving Sales Diversity by Recommending Users to Items

  by Saul Vargas and Pablo Castells

  Sales diversity is considered a key feature of Recommender Systems from a business perspective. Sales diversity is also linked with the long-tail novelty of recommendations, a quality dimension from the user perspective. We explore the inversion of the recommendation task as a means to enhance sales diversity -and indirectly novelty -by selecting which users an item should be recommended to instead of the other way around. We address the inverted task by two approaches: a) inverting the rating matrix, and b) defining a probabilistic reformulation which isolates the popularity component of arbitrary recommendation algorithms. We find that the first approach gives rise to interesting reformulations of nearest-neighbor algorithms, which essentially introduce a new neighbor selection policy. The second approach, as well as the first, ultimately result in substantial sales diversity enhancements, and improved trade-offs with recommendation precision and novelty. Two experiments on movie and music recommendation datasets show the effectiveness of the resulting approach, even when compared to direct optimization approaches of the target metrics proposed in prior work.

• On Over-Specialization and Concentration Biases of Recommendations: Probabilistic Neighborhood Selection in Collaborative Filtering Systems

  by Panagiotis Adamopoulos and Alexander Tuzhilin

  Focusing on the problems of over-specialization and concentration bias, this paper presents a novel probabilistic method for recommending items in the neighborhood-based collaborative filtering framework. For the probabilistic neighborhood selection phase, we use an efficient method for weighted sampling of k neighbors that takes into consideration the similarity levels between the target user (or item) and the candidate neighbors. We conduct an empirical study showing that the proposed method increases the diversity, dispersion, and mobility of recommendations by selecting diverse sets of neighbors. We also demonstrate that the proposed method outperforms popular methods in terms of item prediction accuracy, utility-based ranking, and other measures, across various experimental settings. This performance improvement is in accordance with ensemble learning theory and the phenomenon of “hubness” in recommender systems.

• User Perception of Differences in Movie Recommendation Algorithms

  by Michael Ekstrand, F. Maxwell Harper, Martijn Willemsen and Joseph Konstan

  Recent developments in user evaluation of recommender systems have brought forth powerful new tools for understanding what makes recommendations effective and useful. We apply these methods to understand how users evaluate recommendation lists for the purpose of selecting an algorithm for finding movies. This paper reports on an experiment in which we asked users to compare lists produced by three common collaborative filtering algorithms on the dimensions of novelty, diversity, accuracy, satisfaction, and degree of personalization, and to select a recommender that they would like to use in the future. We find that satisfaction is negatively dependent on novelty and positively dependent on diversity in this setting, and that satisfaction predicts the user’s final selection. We also compare users’ subjective perceptions of recommendation properties with objective measures of those same characteristics. To our knowledge, this is the first study that applies modern survey design and analysis techniques to a within-subjects, direct comparison study of recommender algorithms.

• Offline and Online Evaluation of News Recommender Systems at swissinfo.ch

  by Florent Garcin, Boi Faltings, Olivier Donatsch, Ayar Alazzawi, Christophe Bruttin and Amr Huber

  We report on the live evaluation of various news recommender systems conducted on the website swissinfo.ch. We demonstrate that there is a major difference between offline and online accuracy evaluations. In an offline setting, recommending most popular stories is the best strategy, while in a live environment this strategy is the poorest. For online setting, context-tree recommender systems which profile the users in real-time improve the click-through rate by up to 35%. The visit length also increases by a factor of 2.5. Our experience holds important lessons for the evaluation of recommender systems with offline data as well as for the use of the click-through rate as a performance indicator.

Back to Program