- PADon’t recommend the obvious: estimate probability ratios
by Roberto Pellegrini (Amazon Development Centre Scotland, United Kingdom), Wenjie Zhao (Amazon Development Centre Scotland, United Kingdom), Iain Murray (Amazon Development Centre Scotland, United Kingdom, University of Edinburgh, United Kingdom)
Sequential recommender systems are becoming widespread in the online retail and streaming industry. These systems are often trained to predict the next item given a sequence of a user’s recent actions, and standard evaluation metrics reward systems that can identify the most probable items that might appear next. However, some recent papers instead evaluate recommendation systems with popularity-sampled metrics, which measure how well the model can find a user’s next item when hidden amongst generally-popular items. We argue that these popularity-sampled metrics are more appropriate for recommender systems, because the most probable items for a user often include generally-popular items. If the probability that a customer will watch Toy Story is not much more probable than for the average customer, then the movie isn’t especially relevant for them and we should not recommend it. This paper shows that optimizing popularity-sampled metrics is closely related to estimating point-wise mutual information (PMI). We propose and compare two techniques to fit PMI directly, which both improve popularity-sampled metrics for state-of-the-art recommender systems. The improvements are large compared to differences between recently-proposed model architectures.
Full text in ACM Digital Library
|
- INRecommending for a Multi-Sided Marketplace with Heterogeneous Contents
by Yuyan Wang (Uber Tech. Inc, United States), Long Tao (Uber Tech. Inc, United States), Xian Xing Zhang (Uber Tech. Inc, United States)
Many online personalization platforms today are recommending heterogeneous contents in a multi-sided marketplace consisting of consumers, merchants and other partners. For a recommender system to be successful in these contexts, it faces two main challenges. First, each side in the marketplace has different and potentially conflicting utilities. Recommending for a multi-sided marketplace therefore entails jointly optimizing multiple objectives with trade-offs. Second, the off-the-shelf recommendation algorithms are not applicable to the heterogeneous content space, where a recommendation item could be an aggregation of other recommendation items. In this work, we develop a general framework for recommender systems in a multi-sided marketplace with heterogeneous and hierarchical contents. We propose a constrained optimization framework with machine learning models for each conflicting objective as inputs, and a probabilistic structural model for users’ engagement patterns on heterogeneous contents. Our proposed structural modeling approach ensures consistent user experience across different levels of aggregation of the contents, and provides levels of transparency to the merchants and content providers. We further develop an efficient optimization solution for ranking and recommendation in large-scale online systems in real time. We implement the framework at one of the largest online food delivery platforms in the world, which is a three-sided marketplace consisting of eaters, restaurant partners and delivery partners. Online experiments demonstrate the effectiveness of our framework in ranking heterogeneous contents and optimizing for the three sides in the marketplace. Our framework has been deployed globally as the main recommendation algorithm for the homepage of the platform.
Full text in ACM Digital Library
|
- PASolving Diversity-Aware Maximum Inner Product Search Efficiently and Effectively
by Kohei Hirata (Osaka University, Japan), Daichi Amagata (Osaka University, Japan), Sumio Fujita (Yahoo Japan Corporation, Japan), Takahiro Hara (Osaka University, Japan)
Maximum inner product search (or ?-MIPS) is a fundamental operation in recommender systems that infer preferable items for users. To support large-scale recommender systems, existing studies designed scalable ?-MIPS algorithms. However, these studies do not consider diversity, although recommending diverse items is important to improve user satisfaction. We therefore formulate a new problem, namely diversity-aware ?-MIPS. In this problem, users can control the degree of diversity in their recommendation lists through a parameter. However, exactly solving this problem is unfortunately NP-hard, so it is challenging to devise an efficient, effective, and practical algorithm for the diversity-aware ?-MIPS problem. This paper overcomes this challenge and proposes IP-Greedy, which incorporates new early termination and skipping techniques into a greedy algorithm. We conduct extensive experiments on real datasets, and the results demonstrate the efficiency and effectiveness of our algorithm. Also, we conduct a case study of the diversity-aware ?-MIPS problem on a real dataset. We confirm that this problem can make recommendation lists diverse while preserving high inner products of user and item vectors in the lists.
Full text in ACM Digital Library
|
- PARADio – Rank-Aware Divergence Metrics to Measure Normative Diversity in News Recommendations
by Sanne Vrijenhoek (Universiteit van Amsterdam, Netherlands), Gabriel Bénédict (University of Amsterdam, Netherlands), Mateo Gutierrez Granada (RTL Nederland B.V., Netherlands), Daan Odijk (RTL Nederland B.V., Netherlands), Maarten de Rijke (University of Amsterdam, Netherlands)
In traditional recommender system literature, diversity is often seen as the opposite of similarity, and typically defined as the distance between identified topics, categories or word models. However, this interpretation is not expressive of the normative aspect of news diversity, which also accounts for a news organizations’ norms and values. We introduce RADio, a versatile metrics framework to evaluate recommendations according to these normative goals. RADio introduces a rank-aware Jensen Shannon (JS ) divergence. This combination accounts for (i) a user’s decreasing propensity to observe items further down a list and (ii) full distributional shifts as opposed to point estimates. We evaluate RADio’s ability to reflect five normative concepts in news recommendations on the Microsoft News Dataset and six (neural) recommendation algorithms, with the help of our metadata enrichment pipeline. We find that RADio provides insightful normative estimates that can potentially be used to inform news recommender system design.
Full text in ACM Digital Library
|
- PAReducing Cross-Topic Political Homogenization in Content-Based News Recommendation
by Karthik Shivaram (Tulane University, United States), Ping Liu (Illinois Institute of Technology, United States), Matthew Shapiro (Illinois Institute of Technology, United States), Mustafa Bilgic (Illinois Institute of Technology, United States), Aron Culotta (Tulane University, United States)
Content-based news recommenders learn words that correlate with user engagement and recommend articles accordingly. This can be problematic for users with diverse political preferences by topic — e.g., users that prefer conservative articles on one topic but liberal articles on another. In such instances, recommenders can have a homogenizing effect by recommending articles with the same political lean on both topics, particularly if both topics share salient, politically polarized terms like “far right” or “radical left.” In this paper, we propose attention-based neural network models to reduce this homogenization effect by increasing attention on words that are topic-specific while decreasing attention on polarized, topic-general terms. We find that the proposed approach results in more accurate recommendations for simulated users with such diverse preferences.
Full text in ACM Digital Library
|
- PAExploiting Negative Preference in Content-based Music Recommendation with Contrastive Learning
by Minju Park (Seoul National University, Korea, Republic of), Kyogu Lee (Seoul National University, Korea, Republic of)
Music recommendation systems are in advance nowadays, along with the development of machine learning. At the same time, however, more complex models are being used for music recommendation and they are bringing difficulties of transparency and explainability. In order to give explainability to music recommendation systems, a certain understanding of users’ music tastes must be based on. Meanwhile, there are several studies related to music recommendation systems exploiting negative preference. They have shown improvements in performance, but there was a lack of explanation why negative preference led to better recommendations. In this work, we analyze the role of negative preference in users’ music tastes by comparing music recommendation models with a contrastive objective but with three different training strategies – exploiting preferences of both positive and negative, positive only, and negative only. We evaluate the effectiveness of the negative preference by validating each system with a small amount of personalized data obtained via survey and furthermore propose a method of utilizing them for music recommendation. Our experimental results show that the model exploiting negative preference outperform the other two in terms of accuracy and false positive rate. Furthermore, the proposed training strategies produced a consistent tendency regardless of different types of front-end musical feature extractors, proving the stability of the proposed method.
Full text in ACM Digital Library
|
