Session 17: Women in RecSys

Date: Thursday October 17, 17:10 PM – 18:10 PM (GMT+2)
Room: Petruzzelli Theater
Session Chair: Özlem Özgöbek

• Junior + Senior 🕓10Reproducing Popularity Bias in Recommendation: The Effect of Evaluation Strategies
  by Savvina Daniil (CWI Amsterdam), Mirjam Cuper, Cynthia C. S. Liem, Jacco van Ossenbruggen and Laura Hollink (CWI Amsterdam)

  The extent to which popularity bias is propagated by media recommender systems is a current topic within the community, as is the uneven propagation among users with varying interests for niche items. Recent work focused on exactly this topic, with movies being the domain of interest. Later on, two different research teams reproduced the methodology in the domains of music and books, respectively. The results across the different domains diverge. In this paper, we reproduce the three studies and identify four aspects that are relevant in investigating the differences in results: data, algorithms, division of users in groups and evaluation strategy. We run a set of experiments in which we measure general popularity bias propagation and unfair treatment of certain users with various combinations of these aspects. We conclude that all aspects account to some degree for the divergence in results, and should be carefully considered in future studies. Further, we find that the divergence in findings can be in large part attributed to the choice of evaluation strategy.

• Junior 🕓10Evaluation Measures of Individual Item Fairness for Recommender Systems: A Critical Study
  by Theresia Veronika Rampisela (University of Copenhagen), Maria Maistro, Tuukka Ruotsalo and Christina Lioma

  Fairness is an emerging and challenging topic in recommender systems. In recent years, various ways of evaluating and therefore improving fairness have emerged. In this study, we examine existing evaluation measures of fairness in recommender systems. Specifically, we focus solely on exposure-based fairness measures of individual items that aim to quantify the disparity in how individual items are recommended to users, separate from item relevance to users. We gather all such measures and we critically analyse their theoretical properties. We identify a series of limitations in each of them, which collectively may render the affected measures hard or impossible to interpret, to compute, or to use for comparing recommendations. We resolve these limitations by redefining or correcting the affected measures, or we argue why certain limitations cannot be resolved. We further perform a comprehensive empirical analysis of both the original and our corrected versions of these fairness measures, using real-world and synthetic datasets. Our analysis provides novel insights into the relationship between measures based on different fairness concepts, and different levels of measure sensitivity and strictness. We conclude with practical suggestions of which fairness measures should be used and when. Our code is publicly available. To our knowledge, this is the first critical comparison of individual item fairness measures in recommender systems.

• Junior 🕓10Interactive Content Diversity and User Exploration in Online Movie Recommenders: A Field Experiment
  by Ruixuan Sun (University of Minnesota), Avinash Akella, Ruoyan Kong, Moyan Zhou and Joseph Konstan

  Recommender systems often struggle to strike a balance between matching users’ tastes and providing unexpected recommendations. When recommendations are too narrow and fail to cover the full range of users’ preferences, the system is perceived as useless. Conversely, when the system suggests too many items that users don’t like, it is considered impersonal or ineffective. To better understand user sentiment about the breadth of recommendations given by a movie recommender, we conducted interviews and surveys and found out that many users considered narrow recommendations to be useful, while a smaller number explicitly wanted greater breadth. Additionally, we designed and ran an online field experiment with a larger user group, evaluating two new interfaces designed to provide users with greater access to broader recommendations. We looked at user preferences and behavior for two groups of users: those with higher initial movie diversity and those with lower diversity. Among our findings, we discovered that different level of exploration control and users’ subjective preferences on interfaces are more predictive of their satisfaction with the recommender.

• Senior 🕓10Understanding the Contribution of Recommendation Algorithms on Misinformation Recommendation and Misinformation Dissemination on Social Networks
  by Royal Pathak, Francesca Spezzano and Maria Soledad Pera (TU Delft)

  Social networks are a platform for individuals and organizations to connect with each other and inform, advertise, spread ideas, and ultimately influence opinions. These platforms have been known to propel misinformation. We argue that this could be compounded by the recommender algorithms that these platforms use to suggest items potentially of interest to their users, given the known biases and filter bubbles issues affecting recommender systems. While much has been studied about misinformation on social networks, the potential exacerbation that could result from recommender algorithms in this environment is in its infancy. In this manuscript, we present the result of an in-depth analysis conducted on two datasets (Politifact FakeNewsNet dataset and HealthStory FakeHealth dataset) in order to deepen our understanding of the interconnection between recommender algorithms and misinformation spread on Twitter. In particular, we explore the degree to which well-known recommendation algorithms are prone to be impacted by misinformation. Via simulation, we also study misinformation diffusion on social networks, as triggered by suggestions produced by these recommendation algorithms. Outcomes from this work evidence that misinformation does not equally affect all recommendation algorithms. Popularity-based and network-based recommender algorithms contribute the most to misinformation diffusion. Users who are known to be superspreaders are known to directly impact algorithmic performance and misinformation spread in specific scenarios. Findings emerging from our exploration result in a number of implications for researchers and practitioners to consider when designing and deploying recommender algorithms in social networks.

• Senior 🕓10Where are the values? A systematic literature review on news recommender systems
  by Christine Bauer (Paris Lodron University Salzburg), Chandni Bagchi, Olusanmi Hundogan, and Karin van Es

  In the recommender systems field, it is increasingly recognized that focusing on accuracy measures is limiting and misguided. Unsurprisingly, in recent years, the field has witnessed more interest in the research of values “beyond accuracy.” This trend is particularly pronounced in the news domain where recommender systems perform parts of the editorial function, required to uphold journalistic values of news organizations. In the literature, various values and approaches have been proposed and evaluated. This article reviews the current state of the proposed news recommender systems (NRS). We perform a systematic literature review, analyzing 183 papers. The primary aim is to study the development, scope, and focus of value-aware NRS over time. In contrast to previous surveys, we are particularly interested in identifying the range of values discussed and evaluated in the context of NRS and embrace an interdisciplinary view. We identified a total of 40 values, categorized into five value groups. Most research on value-aware NRS has taken an algorithmic approach, whereas conceptual discussions are comparably scarce. Often, algorithms are evaluated by accuracy-based metrics, but the values are not evaluated with respective measures. Overall, our work identifies research gaps concerning values that have not received much attention. Values need to be targeted on a more fine-grained and specific level.

Back to program