Paper Session 4: User in the Loop

Date: Saturday, Sept 17, 2016, 16:20-18:00
Location: Stratton Student Center (Sala 202)
Chair: Ido Guy

• PPFHCI for Recommender Systems: the Past, the Present and the Future
  by André Calero Valdez, Martina Ziefle, Katrien Verbert

  How can you discover something new, that matches your interest? Recommender Systems have been studied since the 90ies. Their benefit comes from guiding a user through the density of the information jungle to useful knowledge clearings. Early research on recommender systems focuses on algorithms and their evaluation to improve recommendation accuracy using F-measures and other methodologies from signal-detection theory. Present research includes other aspects such as human factors that affect the user experience and interactive visualization techniques to support transparency of results and user control. In this paper, we analyze all publications on recommender systems from the scopus database, and particularly also papers with such an HCI focus. Based on an analysis of these papers, future topics for recommender systems research are identified, which include more advanced support for user control, adaptive interfaces, affective computing and applications in high risk domains.

  Full text in ACM Digital Library

• PPFHuman-Recommender Systems: From Benchmark Data to Benchmark Cognitive Models
  by Patrick Shafto, Olfa Nasraoui

  We bring to the fore of the recommender system research community, an inconvenient truth about the current state of understanding how recommender system algorithms and humans influence one another, both computationally and cognitively. Unlike the great variety of supervised machine learning algorithms which traditionally rely on expert input labels and are typically used for decision making by an expert, recommender systems specifically rely on data input from non-expert or casual users and are meant to be used directly by these same non-expert users on an every day basis. Furthermore, the advances in online machine learning, data generation, and predictive model learning have become increasingly interdependent, such that each one feeds on the other in an iterative cycle. Research in psychology suggests that people’s choices are (1) contextually dependent, and (2) dependent on interaction history. Thus, while standard methods of training and assessing performance of recommender systems rely on benchmark datasets, we suggest that a critical step in the evolution of recommender systems is the development of benchmark models of human behavior that capture contextual and dynamic aspects of human behavior. It is important to emphasize that even extensive real life user-tests may not be sufficient to make up for this gap in benchmarking validity because user tests are typically done with either a focus on user satisfaction or engagement (clicks, sales, likes, etc) with whatever the recommender algorithm suggests to the user, and thus ignore the human cognitive aspect. We conclude by highlighting the interdisciplinary implications of this endeavor.

  Full text in ACM Digital Library

• LPGaze Prediction for Recommender Systems
  by Qian Zhao, Shuo Chang, F. Maxwell Harper, Joseph A. Konstan

  As users browse a recommender system, they systematically consider or skip over much of the displayed content. It seems obvious that these eye gaze patterns contain a rich signal concerning these users’ preferences. However, because eye tracking data is not available to most recommender systems, these signals are not widely incorporated into personalization models. In this work, we show that it is possible to predict gaze by combining easily-collected user browsing data with eye tracking data from a small number of users in a grid-based recommender interface. Our technique is able to leverage a small amount of eye tracking data to infer gaze patterns for other users. We evaluate our prediction models in MovieLens — an online movie recommender system. Our results show that incorporating eye tracking data from a small number of users significantly boosts accuracy as compared with only using browsing data, even though the eye-tracked users are different from the testing users (e.g. AUC=0.823 vs. 0.693 in predicting whether a user will fixate on an item). We also demonstrate that Hidden Markov Models (HMMs) can be applied in this setting; they are better than linear models in predicting fixation probability and capturing the interface regularity through Bayesian inference (AUC=0.823 vs. 0.757).

  Full text in ACM Digital Library

• SPExploring the Value of Personality in Predicting Rating Behaviors: A Study of Category Preferences on MovieLens
  by Raghav Pavan Karumur, Tien T. Nguyen, Joseph A. Konstan

  Prior work relevant to incorporating personality into recommender systems falls into two categories: social science studies and algorithmic ones. Social science studies of preference have found only small relationships between personality and category preferences, whereas, algorithmic approaches found a little improvement when incorporating personality into recommendations. As a result, despite good reasons to believe personality assessments should be useful in recommenders, we are left with no substantial demonstrated impact. In this work, we start with user data from a live recommender system, but study category-by-category variations in preference (both rating levels and distribution) across different personality types. By doing this, we hope to isolate specific areas where personality is most likely to provide value in recommender systems, while also modeling an analytic process that can be used in other domains. After controlling for the family-wise error rate, we find that High Agreeableness users rate at least 0.5 stars higher on a 5-star scale compared to low Agreeableness users. We also find differences in consumption in four different personality types between people who manifested high and low levels of that personality.

  Full text in ACM Digital Library

• SPPairwise Preferences Based Matrix Factorization and Nearest Neighbor Recommendation Techniques
  by Saikishore Kalloori, Francesco Ricci, Marko Tkalcic

  Many recommendation techniques rely on the knowledge of preferences data in the form of ratings for items. In this paper, we focus on pairwise preferences as an alternative way for acquiring user preferences and building recommendations. In our scenario, users provide pairwise preference scores for a set of item pairs, indicating how much one item in each pair is preferred to the other. We propose a matrix factorization (MF) and a nearest neighbor (NN) prediction techniques for pairwise preference scores. Our MF solution maps users and items pairs to a joint latent features vector space, while the proposed NN algorithm leverages specific user-to-user similarity functions well suited for comparing users preferences of that type. We compare our approaches to state of the art solutions and show that our solutions produce more accurate pairwise preferences and ranking predictions.

  Full text in ACM Digital Library

• SPObserving Group Decision Making Processes
  by Amra Delic, Julia Neidhardt, Thuy Ngoc Nguyen, Francesco Ricci, Laurens Rook, Hannes Werthner, Markus Zanker

  Most research on group recommender systems relies on the assumption that individuals have conflicting preferences; in order to generate group recommendations the system should identify a fair way of aggregating these preferences. Both empirical studies and theoretical frameworks have tried to identify the most effective preference aggregation techniques without coming to definite conclusions. In this paper, we propose to approach group recommendation from the group dynamics perspective and analyze the group decision making process for a particular task (in the travel domain). We observe several individual and group properties and correlate them to choice satisfaction. Supported by these initial results we therefore advocate for the development of new group recommendation techniques that consider group dynamics and support the full group decision making process.

  Full text in ACM Digital Library

• SP BPNExpLOD: A Framework for Explaining Recommendations based on the Linked Open Data Cloud
  by Cataldo Musto, Fedelucio Narducci, Pasquale Lops, Marco De Gemmis, Giovanni Semeraro

  In this paper we present ExpLOD, a framework which exploits the information available in the Linked Open Data (LOD) cloud to generate a natural language explanation of the suggestions produced by a recommendation algorithm. The methodology is based on building a graph in which the items liked by a user are connected to the items recommended through the properties available in the LOD cloud. Next, given this graph, we implemented some techniques to rank those properties and we used the most relevant ones to feed a module for generating explanations in natural language. In the experimental evaluation we performed a user study with 308 subjects aiming to investigate to what extent our explanation framework can lead to more transparent, trustful and engaging recommendations. The preliminary results provided us with encouraging findings, since our algorithm performed better than both a non-personalized explanation baseline and a popularity-based one.

  Full text in ACM Digital Library

• SPThe Value of Online Customer Reviews
  by Georgios Askalidis, Edward C. Malthouse

  We study the effect of the volume of consumer reviews on the purchase likelihood (conversion rate) of users browsing a product page. We propose using the exponential learning curve model to study how conversion rates change with the number of reviews. We call the difference in conversion rate between having no reviews and an infinite number the value of reviews. We find that, on average, the conversion rate of a product can increase by as much as 270% as it accumulates reviews, amongst the users that choose to display them. We also find diminishing marginal value as a product accumu- lates reviews, with the first five reviews driving the bulk of the aforementioned increase. To address the problem of si- multaneity of increase of reviews and conversion rate, we use customer sessions in which reviews were not displayed as a control for trends that would have happened regardless of the increase in the review volume. Using our framework, we further find that high priced items have a higher value for reviews than lower priced items. High priced items can see their conversion rate increase by as much as 380% as they accumulate reviews compared to 190% for low priced items.We infer that the existence of reviews provides valu- able signals to the customers, increasing their propensity to purchase. We also infer that users usually don’t pay atten- tion to the entire set of reviews, especially if there are a lot of them, but instead they focus on the first few available. Our approach can be extended and applied in a variety of settings to gain further insights.

  Full text in ACM Digital Library

Back to Program