Paper Session 1: Why Did I Get This? Explaining Recommendations

Date: Wednesday, Oct 3, 2018, 11:00-12:30
Location: Parq D/E/F
Chair: Nava Tintarev

  • LPEffects of Personal Characteristics on the Music Recommender with Different Controllability
    by Yucheng Jin, Nava Tintarev, Katrien Verbert

    Previous research has found that enabling users to control the recommendation process increases user satisfaction with recommendations. However, providing additional controls also increases cognitive load, and different users have different needs for control. Therefore, in this study, we investigate the effect of two personal characteristics: musical sophistication and visual memory capacity. We designed a visual user interface, on top of a commercial music recommender, that incorporates different controls: interactions with recommendations (i.e., the output of a recommender system), the user profile (i.e., the top listened songs), and algorithm parameters (i.e., weights in an algorithm). We created eight experimental settings with all possible combinations of these three user controls and conducted a between-subjects study (N=240), to explore how these controls influence cognitive load and recommendation acceptance for different personal characteristics. We found that controlling recommendations is the most favorable single control element. In addition, controlling recommendations and algorithm parameters is the most beneficial setting with multiple controls. Moreover, the participants with high musical sophistication are more likely to accept recommendations, suggesting that they perceive recommendations to be of higher quality. However, we found no effect of visual working memory on either cognitive load or recommendation acceptance. This work contributes an understanding of how to design control that hit the sweet spot between the perceived quality of recommendations and acceptable cognitive load.

    Full text in ACM Digital Library

  • LPProviding Explanations for Recommendations in Reciprocal Environments
    by Akiva Kleinerman, Rosenfeld Ariel, Sarit Kraus

    Automated platforms which support users in finding a mutually beneficial match, such as online dating and job recruitment sites, are becoming increasingly popular. These platforms often include recommender systems that assist users in finding a suitable match. While recommender systems which provide explanations for their recommendations have shown many benefits, explanation methods have yet to be adapted and tested in recommending suitable matches. In this paper, we introduce and extensively evaluate the use of reciprocal explanations– explanations which provide reasoning as to why both parties are expected to benefit from the match. Through an extensive empirical evaluation, in both simulated and real-world dating platforms with 287 human participants, we find that when the acceptance of a recommendation involves a significant cost (e.g., monetary or emotional), reciprocal explanations outperform standard explanation methods which consider the recommendation receiver alone. However, to the contrary of to what one may expect, when the cost of accepting a recommendation is negligible, reciprocal explanations are shown to be less effective than the traditional explanation methods.

    Full text in ACM Digital Library

  • LPExplore, Exploit, and Explain: Personalizing Explainable Recommendations with Bandits
    by James McInerney, Benjamin Lacker, Samantha Hansen, Karl Higley, Hugues Bouchard, Alois Gruson, Rishabh Mehrotra

    The multi-armed bandit is an important framework for balancing exploration with exploitation in recommendation. Exploitation recommends content (e.g., products, movies, music playlists) with the highest predicted user engagement and has traditionally been the focus of recommender systems. Exploration recommends content with uncertain predicted user engagement for the purpose of gathering more information. The importance of exploration has been recognized in recent years, particularly in settings with new users, new items, non-stationary preferences and attributes. In parallel, explaining recommendations (“recsplanations”) is crucial if users are to understand their recommendations. Existing work has looked at bandits and explanations independently. We provide the first method that combines both in a principled manner. In particular, our method is able to jointly (1) learn which explanations each user responds to; (2) learn the best content to recommend for each user; and (3) balance exploration with exploitation to deal with uncertainty. Experiments with historical log data and tests with live production traffic in a large-scale music recommendation service show a significant improvement in user engagement.

    Full text in ACM Digital Library

  • LPInterpreting User Inaction in Recommender Systems
    by Qian Zhao, Martijn Willemsen, Gediminas Adomavicius, F. Maxwell Harper, Joseph A. Konstan

    Temporally, users browse and interact with items in recommender systems. However, for most systems, the majority of the displayed items do not elicit any action from users. In other words, the user-system interaction process includes three aspects: browsing, action, and inaction. Prior recommender systems literature has focused more on actions than on browsing or inaction. In this work, we deployed a field survey in a live movie recommender system to interpret what inaction means from both the user’s and the system’s perspective, guided by psychological theories of human decision making. We further systematically study factors to infer the reasons of user inaction and demonstrate with offline data sets that this descriptive and predictive inaction model can provide benefits for recommender systems in terms of both action prediction and recommendation timing.

    Full text in ACM Digital Library

  • SPOImpact of Item Consumption on Assessment of Recommendations in User Studies
    by Benedikt Loepp, Tim Donkers, Timm Kleemann, Jürgen Ziegler

    In user studies of recommender systems, participants typically cannot consume the recommended items. Still, they are asked to assess recommendation quality and other aspects related to user experience by means of questionnaires. Without having listened to recommended songs or watched suggested movies, however, this might be an error-prone task, possibly limiting validity of results obtained in these studies. In this paper, we investigate the effect of actually consuming the recommended items. We present two user studies conducted in different domains showing that in some cases, differences in the assessment of recommendations and in questionnaire results occur. Apparently, it is not always possible to adequately measure user experience without allowing users to consume items. On the other hand, depending on domain and provided information, participants sometimes seem to approximate the actual value of recommendations reasonably well.

    Full text in ACM Digital Library

Back to Program

Diamond Supporter
Platinum Supporters
Gold Supporters
Silver Supporters
Special Supporter