Tuesday Posters

Date: Tuesday October 15
Room: Chamber of Commerce

• DEMOA Tool for Explainable Pension Fund Recommendations using Large Language Models
  by Eduardo Alves da Silva (IComp – Institute of Computing, Federal University of Amazonas; University of Vale do Itajaí; Saks Global), Leandro Balby Marinho (Federal University of Campina Grande), Edleno Silva de Moura (IComp – Institute of Computing, Federal University of Amazonas) and Altigran Soares da Silva (IComp – Institute of Computing, Federal University of Amazonas)

  In this demo, we present a prototype tool designed to help financial advisors recommend private pension funds to investors based on their preferences, offering personalized investment suggestions. The tool leverages Large Language Models (LLMs), which enhance explainability by providing clear and understandable rationales for recommendations and effectively handles both sequential and cold-start scenarios. We outline the design, implementation, and results of a user-based evaluation using real-world data. The evaluation shows a high recommendation acceptance rate among financial advisors, highlighting the tool’s potential to improve decision-making in financial advisory services.

  Full text in ACM Digital Library

• INDAI-assisted Coding with Cody: Lessons from Context Retrieval and Evaluation for Code Recommendations
  by Jan Hartman (Sourcegraph), Hitesh Sagtani (Sourcegraph), Julie Tibshirani (Sourcegraph) and Rishabh Mehrotra (Sourcegraph)

  In this work, we discuss a recently popular type of recommender system: an LLM-based coding assistant. Connecting the task of providing code recommendations in multiple formats to traditional RecSys challenges, we outline several similarities and differences due to domain specifics. We emphasize the importance of providing relevant context to an LLM for this use case and discuss lessons learned from context enhancements & offline and online evaluation of such AI-assisted coding systems.

  Full text in ACM Digital Library

• INDAnalyzing User Preferences and Quality Improvement on Bing’s WebPage Recommendation Experience with Large Language Models
  by Jaidev Shah (Microsoft AI), Gang Luo (Microsoft AI), Jialin Liu (Microsoft AI), Amey Barapatre (Microsoft AI), Fan Wu (Microsoft AI), Chuck Wang (Microsoft AI) and Hongzhi Li (Microsoft AI)

  Explore Further @ Bing (Web Recommendations) is a web-scale query independent webpage-to-webpage recommendation system with an index size of over 200 billion webpages. Due to the significant variability in webpage quality across the web and the reliance of our system on learning soleley user behavior (clicks), our production system was susceptible to serving clickbait and low-quality recommendations. Our team invested several months in developing and shipping several improvements that utilize LLM-generated recommendation quality labels to enhance our ranking stack to improve the nature of the recommendations we show to our users. Another key motivation behind our efforts was to go beyond merely surfacing relevant webpages, focusing instead on prioritizing more useful and authoritative content that delivers value to users based on their implied intent. We demonstrate how large language models (LLMs) offer a powerful tool for product teams to gain deeper insights into shifts in product experience and user behavior following significant improvements or changes to a production system. In this work, to enable our analysis, we also showcase the use of a small language model (SLM) to generate better-quality webpage text features and summaries at scale and describe our approach to mitigating position bias in user interaction logs.”

  Full text in ACM Digital Library

• LBRbeeFormer: Bridging the Gap Between Semantic and Interaction Similarity in Recommender Systems
  by Vojtěch Vančura (Czech Technical University), Pavel Kordík (Czech Technical University) and Milan Straka (Charles University)

  Recommender systems often use text-side information to improve their predictions, especially in cold-start or zero-shot recommendation scenarios, where traditional collaborative filtering approaches cannot be used. Many approaches to text-mining side information for recommender systems have been proposed over recent years, with sentence Transformers being the most prominent one. However, these models are trained to predict semantic similarity without utilizing interaction data with hidden patterns specific to recommender systems. In this paper, we propose beeFormer, a framework for training sentence Transformer models with interaction data. We demonstrate that our models trained with beeFormer can transfer knowledge between datasets while outperforming not only semantic similarity sentence Transformers but also traditional collaborative filtering methods. We also show that training on multiple datasets from different domains accumulates knowledge in a single model, unlocking the possibility of training universal, domain-agnostic sentence Transformer models to mine text representations for recommender systems. We release the source code, trained models, and additional details allowing replication of our experiments at https://github.com/recombee/beeformer.

  Full text in ACM Digital Library

• RESBetter Generalization with Semantic IDs: A Case Study in Ranking for Recommendations
  by Anima Singh (Google), Trung Vu (Google), Nikhil Mehta (Google DeepMind), Raghunandan Keshavan (Google), Maheswaran Sathiamoorthy (Google DeepMind), Yilin Zheng (Google), Lichan Hong (Google DeepMind), Lukasz Heldt (Google), Li Wei (Google), Devansh Tandon (Google), Ed Chi (Google DeepMind) and Xinyang Yi (Google DeepMind)

  Randomly-hashed item ids are used ubiquitously in recommendation models. However, the learned representations from random hashing prevents generalization across similar items, causing problems of learning unseen and long-tail items, especially when item corpus is large, power-law distributed, and evolving dynamically. In this paper, we propose using content-derived features as a replacement for random ids. We show that simply replacing ID features with content-based embeddings can cause a drop in quality due to reduced memorization capability. To strike a good balance of memorization and generalization, we propose to use Semantic IDs, a compact and discrete item representation, as a replacement for random item ids. Semantic IDs are learned from frozen content embeddings using RQ-VAE and thus can capture the hierarchy of concepts in items. Similar to content embeddings, the compactness of Semantic IDs poses a problem of adaption in recommendation models. We propose novel methods for adapting Semantic IDs in industry-scale ranking models, through hashing sub-pieces of of the Semantic-ID sequences. In particular, we find that the SentencePiece model that is commonly used in LLM tokenization outperforms manually crafted pieces such as N-grams. To the end, we evaluate our approaches in a real-world ranking model for YouTube recommendations. Our experiments demonstrate that Semantic IDs can replace the direct use of video IDs by improving the generalization ability on new and long-tail item slices without sacrificing overall model quality.

  Full text in ACM Digital Library

• INDCo-optimize Content Generation and Consumption in a Large Scale Video Recommendation System
  by Zhen Zhang (Google Inc.), Qingyun Liu (Google DeepMind), Yuening Li (Google Inc.), Sourabh Bansod (Google Inc.), Mingyan Gao (Google Inc.), Yaping Zhang (Google Inc.), Zhe Zhao (Google DeepMind), Lichan Hong (Google DeepMind), Ed H. Chi (Google DeepMind), Shuchao Bi (Google Inc.) and Liang Liu (Google Inc.)

  Multi-task prediction models and value models are the de-facto standard ranking components in modern large-scale content recommendation systems. However, they are typically optimized to model users’ passive consumption behaviors, and rank content in a way to grow only consumption-centric values. In this talk, we discuss the key insight that it is possible to model sparse participatory content-generation actions as well and grow ecosystem value through a new ranking system. We made the following key technical contributions in this system: (1) introducing ranking for content generation based on a categorization of user participation actions of different sparsity, including proxy intent action or access point clicks. (2) improving sparse task prediction quality and stability by causal task relationship modeling, conditional loss modeling and ResNet based shared bottom network. (3) personalizing the value model to minimize conflicts between different values, through e.g. ranking inspiring content higher for users who actively generate content. (4) conducting systematic evaluation of proposed approach in a large short-form video UGC (User-Generated Content) platform.

  Full text in ACM Digital Library

• RESData Augmentation using Reverse Prompt for Cost-Efficient Cold-Start Recommendation
  by Genki Kusano (NEC)

  Recommendation systems that use auxiliary information such as product names and categories have been proposed to address the cold-start problem. However, these methods do not perform well when we only have insufficient warm-start training data. On the other hand, large language models (LLMs) can perform as effective cold-start recommendation systems even with limited warm-start data. However, they require numerous API calls for inferences, which leads to high operational costs in terms of time and money. This is a significant concern in industrial applications. In this paper, we introduce a new method, RevAug, which leverages LLMs as a data augmentation to enhance cost-efficient cold-start recommendation systems. To generate pseudo-samples, we have reversed the commonly used prompt for an LLM from “Would this user like this item?” to “What kind of items would this user like?”. Generated outputs by this reverse prompt are pseudo-auxiliary information utilized to enhance recommendation systems in the training phase. In numerical experiments with four real-world datasets, RevAug demonstrated superior performance in cold-start settings with limited warm-start data compared to existing methods. Moreover, RevAug significantly reduced API fees and processing time compared to an LLM-based recommendation method.

  Full text in ACM Digital Library

• LBRDemocratizing Urban Mobility Through an Open-Source, Multi-Criteria Route Recommendation System
  by Alexander Eggerth (ETH Zurich), Javier Argota Sánchez-Vaquerizo (ETH Zurich), Dirk Helbing (ETH Zurich) and Sachit Mahajan (ETH Zurich)

  Urban navigation systems traditionally optimize for efficiency, thereby overlooking environmental factors and personal preferences. This paper introduces Routify, a novel multi-criteria recommender system for personalized urban route selection. Routify advances the state-of-the-art in route recommendation through: (1) integration of diverse environmental data (air quality, noise levels, green spaces) with real-time IoT inputs; (2) a dynamic weighting algorithm allowing fine-grained user control over routing parameters; (3) transparent decision-making via an interactive interface displaying edge-level information; and (4) a community-driven feedback mechanism that continuously refines recommendations. We have implemented Routify as an open-source platform, facilitating further research and development. Experimental results from 1000 origin-destination pairs demonstrate significant improvements in user-defined environmental metrics (up to 32.98% increase in green index for walking and 16.91% reduction in noise levels for cycling) compared to traditional routing systems. These improvements come with slight trade-offs in distance, especially in walking mode. Our research contributes a comprehensive framework for multi-criteria route recommendation, balancing individual preferences, environmental factors, and community insights in urban navigation.

  Full text in ACM Digital Library

• RESDo Not Wait: Learning Re-Ranking Model Without User Feedback At Serving Time in E-Commerce
  by Yuan Wang (Alibaba Group), Zhiyu Li (Alibaba Group), Changshuo Zhang (Renmin University of China), Sirui Chen (Renmin University of China), Xiao Zhang (Renmin University of China), Jun Xu (Renmin University of China) and Quan Lin (Alibaba Group)

  Recommender systems have been widely used in e-commerce, and re-ranking models are playing an increasingly significant role in the domain, which leverages the inter-item influence and determines the final recommendation lists. Online learning methods keep updating a deployed model with the latest available samples to capture the shifting of the underlying data distribution in e-commerce. However, they depend on the availability of real user feedback, which may be delayed by hours or even days, such as item purchases, leading to a lag in model enhancement. In this paper, we propose a novel extension of online learning methods for re-ranking modeling, which we term LAST, an acronym for Learning At Serving Time. It circumvents the requirement of user feedback by using a surrogate model to provide the instructional signal needed to steer model improvement. Upon receiving an online request, LAST finds and applies a model modification on the fly before generating a recommendation result for the request. The modification is request-specific and transient. It means the modification is tailored to and only to the current request to capture the specific context of the request. After a request, the modification is discarded, which helps to prevent error propagation and stabilizes the online learning procedure since the predictions of the surrogate model may be inaccurate. Most importantly, as a complement to feedback-based online learning methods, LAST can be seamlessly integrated into existing online learning systems to create a more adaptive and responsive recommendation experience. Comprehensive experiments, both offline and online, affirm that LAST outperforms state-of-the-art re-ranking models.

  Full text in ACM Digital Library

• INDDynamic Product Image Generation and Recommendation at Scale for Personalized E-commerce
  by Ádám Tibor Czapp (Taboola Budapest), Mátyás Jani (Taboola Budapest), Bálint Domián (Taboola Budapest) and Balázs Hidasi (Taboola Budapest)

  Coupling latent diffusion based image generation with contextual bandits enables the creation of eye-catching personalized product images at scale that was previously either impossible or too expensive. In this paper we showcase how we utilized these technologies to increase user engagement with recommendations in online retargeting campaigns for e-commerce.

  Full text in ACM Digital Library

• RESEfficient Inference of Sub-Item Id-based Sequential Recommendation Models with Millions of Items
  by Aleksandr Vladimirovich Petrov (University of Glasgow), Craig Macdonald (University of Glasgow) and Nicola Tonellotto (University of Pisa)

  Transformer-based recommender systems, such as BERT4Rec or SASRec, achieve state-of-the-art results in sequential recommendation. However, it is challenging to use these models in production environments with catalogues of millions of items: scaling Transformers beyond a few thousand items is problematic for several reasons, including high model memory consumption and slow inference. In this respect, RecJPQ is a state-of-the-art method of reducing the models’ memory consumption; RecJPQ compresses item catalogues by decomposing item IDs into a small number of shared sub-item IDs. Despite reporting the reduction of memory consumption by a factor of up to 50 ×, the original RecJPQ paper did not report inference efficiency improvements over the baseline Transformer-based models. Upon analysing RecJPQ’s scoring algorithm, we find that its efficiency is limited by its use of score accumulators for each item, which prevents parallelisation. In contrast, LightRec (a non-sequential method that uses a similar idea of sub-ids) reported large inference efficiency improvements using an algorithm we call PQTopK. We show that it is also possible to improve RecJPQ-based models’ inference efficiency using the PQTopK algorithm. In particular, we speed up RecJPQ-enhanced SASRec by a factor of 4.5 × compared to the original SASRec’s inference method and by the factor of 1.56 × compared to the method implemented in RecJPQ code on a large-scale Gowalla dataset with more than million items. Further, using simulated data, we show that PQTopK remains efficient with catalogues of up to tens of millions of items, removing one of the last obstacles to using Transformer-based models in production environments with large catalogues.

  Full text in ACM Digital Library

• INDEmbedding based retrieval for long tail search queries in ecommerce
  by Akshay Kekuda (Best Buy), Yuyang Zhang (Best Buy) and Arun Udayashankar (Best Buy)

  In this abstract we present a series of optimizations we performed on the two-tower model architecture, training and evaluation datasets to implement semantic product search at Best Buy. Search queries on bestbuy.com follow the pareto distribution whereby a minority of them account for most searches. This leaves us with a long tail of search queries that have low frequency of issuance. The queries in the long tail suffer from very spare interaction signals. Our current work focuses on building a model to serve the long tail queries. We present a series of optimizations we have done to this model to maximize conversion for the purpose of retrieval from the catalog.

  The first optimization we present is using a large language model to improve the sparsity of conversion signals. The second optimization is pretraining an off-the-shelf transformer-based model on the Best Buy catalog data. The third optimization we present is on the finetuning front. We use query-to-query pairs in addition to query-to-product pairs and combining the above strategies for finetuning the model. We also demonstrate how merging the weights of these finetuned models improves the evaluation metrics. Finally, we provide a recipe for curating an evaluation dataset for continuous monitoring of model performance with human-in-the-loop evaluation. We found that adding this recall mechanism to our current term match-based recall improved conversion by 3% in an online A/B test.

  Full text in ACM Digital Library

• INDEncouraging Exploration in Spotify Search through Query Recommendations
  by Henrik Lindstrom (Spotify), Humberto Jesus Corona Pampin (Spotify), Enrico Palumbo (Spotify) and Alva Liu (Spotify)

  At Spotify, search has been traditionally seen as a tool for retrieving content, with the search system optimized for when the user has a specific target in mind. In particular we have relied on an instant search system providing results for each keystroke, which works well for known-item search, when queries are straightforward, and the catalog is small. However, as Spotify’s catalog grows in size and variety, it becomes increasingly difficult for users to define their search intents accurately. Furthermore, as we expand the offering, we need to help users discover more content both when it comes to new content types, e.g. audiobooks, as well as for new content/creators within existing content types. To solve this we have introduced a hybrid Query Recommendation system (QR) that helps the user formulate more complex exploratory search intents, while still serving known-item lookups efficiently. This experience has been rolled out worldwide to all mobile users resulting in an increase in exploratory intent queries of 9% in A/B tests.

  Full text in ACM Digital Library

• RESFairCRS: Towards User-oriented Fairness in Conversational Recommendation Systems
  by Qin Liu (Jinan University), Xuan Feng (Jinan University), Tianlong Gu (Jinan University) and Xiaoli Liu (Jinan University)

  Conversational Recommendation Systems (CRSs) enable recommender systems to explicitly acquire user preferences during multi-turn interactions, providing more accurate and personalized recommendations. However, the data imbalance in CRSs, due to inconsistent interaction history among users, may lead to disparate treatment for disadvantaged user groups. In this paper, we investigate the discriminate problems in CRS from the user’s perspective, called as user-oriented fairness. To reveal the unfairness problems of different user groups in CRS, we conduct extensive empirical analyses. To mitigate user unfairness, we propose a user-oriented fairness framework, named FairCRS, which is a model-agnostic framework. In particular, we develop a user-embedding reconstruction mechanism that enriches user embeddings by incorporating more interaction information, and design a user-oriented fairness strategy that optimizes the recommendation quality differences among user groups while alleviating unfairness. Extensive experimental results on English and Chinese datasets show that FairCRS outperforms state-of-the-art CRSs in terms of overall recommendation performance and user fairness.

  Full text in ACM Digital Library

• DEMOGenUI(ne) CRS: UI Elements and Retrieval-Augmented Generation in Conversational Recommender Systems with LLMs
  by Ulysse Maes (Vrije Universiteit Brussel), Lien Michiels (Vrije Universiteit Brussel; University of Antwerp) and Annelien Smets (Vrije Universiteit Brussel)

  Previous research has used Large Language Models (LLMs) to develop personalized Conversational Recommender Systems (CRS) with text-based user interfaces (UIs). However, the potential of LLMs to generate interactive graphical elements that enhance user experience remains largely unexplored. To address this gap, we introduce “GenUI(ne) CRS,” a novel framework designed to leverage LLMs for adaptive and interactive UIs. Our framework supports domain-specific graphical elements such as buttons and cards, in addition to text-based inputs. It also addresses the common LLM issue of outdated knowledge, known as the “knowledge cut-off,” by implementing Retrieval-Augmented Generation (RAG). To illustrate its potential, we developed a prototype movie CRS. This work demonstrates the feasibility of LLM-powered interactive UIs and paves the way for future CRS research, including user experience validation, transparent explanations, and addressing LLM biases.

  Full text in ACM Digital Library

• LBRInformed Dataset Selection with ‘Algorithm Performance Spaces’
  by Joeran Beel (University of Siegen), Lukas Wegmeth (University of Siegen), Lien Michiels (University of Antwerp) and Steffen Schulz (University of Siegen)

  When designing recommender-systems experiments, a key question that has been largely overlooked is the choice of datasets. In a brief survey of ACM RecSys papers, we found that authors typically justified their dataset choices by labelling them as public, benchmark, or ‘real-world’ without further explanation. We propose the Algorithm Performance Space (APS) as a novel method for informed dataset selection. The APS is an n-dimensional space where each dimension represents the performance of a different algorithm. Each dataset is depicted as an n-dimensional vector, with greater distances indicating higher diversity. In our experiment, we ran 29 algorithms on 95 datasets to construct an actual APS. Our findings show that many datasets, including most Amazon datasets, are clustered closely in the APS, i.e. they are not diverse. However, other datasets, such as MovieLens and Docear, are more dispersed. The APS also enables the grouping of datasets based on the solvability of the underlying problem. Datasets in the top right corner of the APS are considered ’solved problems’ because all algorithms perform well on them. Conversely, datasets in the bottom left corner lack well-performing algorithms, making them ideal candidates for new recommender-system research due to the challenges they present.

  Full text in ACM Digital Library

• LBRIs It Really Complementary? Revisiting Behavior-based Labels for Complementary Recommendation
  by Kai Sugahara (The University of Electro-Communications), Chihiro Yamasaki (The University of Electro-Communications) and Kazushi Okamoto (The University of Electro-Communications)

  Complementary recommendation is a type of item-to-item recommendation that recommends what should be purchased together for an item. Previous studies have traditionally used behavior-based labels (BBLs) that are constructed from the co-purchase logs of users for training and evaluation because rigorous label construction for complements is inefficient. However, the fact that many item pairs in BBLs are not functionally complementary, even though they are frequently co-purchased, has been overlooked. This study aimed to re-evaluate the validity of BBLs through functional relationships and provide directions for their improvement. Quantitative analysis using manually annotated function-based labels (FBLs) as correct labels revealed that the accuracy of the complementary recommendations generated by BBLs was below 50%, suggesting potential functional incompatibility within BBLs. Existing models that were trained on BBLs were similarly inaccurate, indicating the unreliability of the evaluations in existing studies. Finally, we proposed a label correction method for BBLs using a small set of FBLs, thereby providing a direction for reliable complementary recommendations.

  Full text in ACM Digital Library

• RESIt’s Not You, It’s Me: The Impact of Choice Models and Ranking Strategies on Gender Imbalance in Music Recommendation
  by Andres Ferraro (Pandora/SiriusXM), Michael D. Ekstrand (Drexel University) and Christine Bauer (Paris Lodron University Salzburg)

  As recommender systems are prone to various biases, mitigation approaches are needed to ensure that recommendations are fair to various stakeholders. One particular concern in music recommendation is artist gender fairness. Recent work has shown that the gender imbalance in the sector translates to the output of music recommender systems, creating a feedback loop that can reinforce gender biases over time.

  In this work, we examine that feedback loop to study whether algorithmic strategies or user behavior are a greater contributor to ongoing improvement (or loss) in fairness as models are repeatedly re-trained on new user feedback data. We simulate user interaction and re-training to investigate the effects of ranking strategies and user choice models on gender fairness metrics. We find re-ranking strategies have a greater effect than user choice models on recommendation fairness over time.

  Full text in ACM Digital Library

• LBRKGGLM: A Generative Language Model for Generalizable Knowledge Graph Representation Learning in Recommendation
  by Giacomo Balloccu (University of Cagliari), Ludovico Boratto (University of Cagliari), Gianni Fenu (University of Cagliari), Mirko Marras (University of Cagliari) and Alessandro Soccol (University of Cagliari)

  Current recommendation methods based on knowledge graphs rely on entity and relation representations for several steps along the pipeline, with knowledge completion and path reasoning being the most influential. Despite their similarities, the most effective representation methods for these steps differ, leading to inefficiencies, limited representativeness, and reduced interpretability. In this paper, we introduce KGGLM, a decoder-only Transformer model designed for generalizable knowledge representation learning to support recommendation. The model is trained on generic paths sampled from the knowledge graph to capture foundational patterns, and then fine-tuned on paths specific of the downstream step (knowledge completion and path reasoning in our case). Experiments on ML1M and LFM1M show that KGGLM beats twenty-two baselines in effectiveness under both knowledge completion and recommendation. Source code and pre-processed data sets are available at https://github.com/mirkomarras/kgglm.

  Full text in ACM Digital Library

• RESLARR: Large Language Model Aided Real-time Scene Recommendation with Semantic Understanding
  by Zhizhong Wan (Meituan), Bin Yin (Meituan), Junjie Xie (Meituan), Fei Jiang (Meituan), Xiang Li (Meituan) and Wei Lin (Meituan)

  Click-Through Rate (CTR) prediction is crucial for Recommendation System(RS), aiming to provide personalized recommendation services for users in many aspects such as food delivery, e-commerce and so on. However, traditional RS relies on collaborative signals, which lacks semantic understanding to real-time scenes. We also noticed that a major challenge in utilizing Large Language Models (LLMs) for practical recommendation purposes is their efficiency in dealing with long text input. To break through the problems above, we propose Large Language Model Aided Real-time Scene Recommendation(LARR), adopt LLMs for semantic understanding, utilizing real-time scene information in RS without requiring LLM to process the entire real-time scene text directly, thereby enhancing the efficiency of LLM-based CTR modeling. Specifically, recommendation domain-specific knowledge is injected into LLM and then RS employs an aggregation encoder to build real-time scene information from separate LLM’s outputs. Firstly, a LLM is continual pretrained on corpus built from recommendation data with the aid of special tokens. Subsequently, the LLM is fine-tuned via contrastive learning on three kinds of sample construction strategies. Through this step, LLM is transformed into a text embedding model. Finally, LLM’s separate outputs for different scene features are aggregated by an encoder, aligning to collaborative signals in RS, enhancing the performance of recommendation model.

  Full text in ACM Digital Library

• RESLLMs for User Interest Exploration in Large-scale Recommendation Systems
  by Jianling Wang (Google DeepMind), Haokai Lu (Google DeepMind), Yifan Liu (Google), He Ma (Google), Yueqi Wang (Google), Yang Gu (Google), Shuzhou Zhang (Google), Ningren Han (Google), Shuchao Bi (Google), Lexi Baugher (Google), Ed H. Chi (Google DeepMind) and Minmin Chen (Google DeepMind)

  Traditional recommendation systems are subject to a strong feedback loop by learning from and reinforcing past user-item interactions, which in turn limits the discovery of novel user interests. To address this, we introduce a hybrid hierarchical framework combining Large Language Models (LLMs) and classic recommendation models for user interest exploration. The framework controls the interfacing between the LLMs and the classic recommendation models through “interest clusters”, the granularity of which can be explicitly determined by algorithm designers. It recommends the next novel interests by first representing “interest clusters” using language, and employs a fine-tuned LLM to generate novel interest descriptions that are strictly within these predefined clusters. At the low level, it grounds these generated interests to an item-level policy by restricting classic recommendation models, in this case a transformer-based sequence recommender to return items that fall within the novel clusters generated at the high level. We showcase the efficacy of this approach on an industrial-scale commercial platform serving billions of users. Live experiments show a significant increase in both exploration of novel interests and overall user enjoyment of the platform.

  Full text in ACM Digital Library

• RESMAWI Rec: Leveraging Severe Weather Data in Recommendation
  by Brendan Andrew Duncan (UC San Diego), Surya Kallumadi (Lowe’s Companies, Inc.), Taylor Berg-Kirkpatrick (UC San Diego) and Julian Mcauley (University of California San Diego)

  Inferring user intent in recommender systems can help performance but is difficult because intent is personal and not directly observable. Previous work has leveraged signals to stand as a proxy for intent (e.g. user interactions with resource pages), but such signals are not always available. In this paper, we instead recognize that certain events, which are observable, directly influence user intent. For example, after a flood, home improvement customers are more likely to undertake a renovation project to dry out their basement. We introduce MAWI Rec, a recommender system that leverages severe weather data to improve recommendation. Our weather-aware system achieves a significant improvement over a state-of-the-art baseline for online and in-store datasets of home improvement customers. This gain is most significant for weather-related product categories such as roof panels and flashings.

  Full text in ACM Digital Library

• RESMODEM: Decoupling User Behavior for Shared-Account Video Recommendations on Large Screen Devices
  by Jiang Li (University of Science and Technology of China), Zhen Zhang (Kuaishou Technology Co., Ltd.), Xiang Feng (Kuaishou Technology Co., Ltd.), Muyang Li (Kuaishou Technology Co., Ltd.), Yongqi Liu (Kuaishou Technology Co., Ltd.) and Lantao Hu (Kuaishou Technology Co., Ltd.)

  In scenarios involving sequence recommendations on large screen devices, such as tablets or TVs, the equipment is often shared among multiple users. This sharing leads to a mixture of behaviors from different users, posing significant challenges to recommendation systems, especially when clear supervisory signals for distinguishing among users are absent. Current solutions tend to either operate in an unsupervised manner or rely on constructed supervisory signals that are not entirely reliable. Moreover, the peculiarities of short video recommendations in this context have not been thoroughly explored in existing research. In response to these challenges, this paper introduces Multi-User Contrastive Decoupling Model (MODEM), a novel short video recommendation model specifically designed for large screen devices. MODEM leverages an attention mechanism, grounded in session segmentation, to disentangle the intertwined user behavior histories. It also discriminates between the impacts of long and short viewing behaviors on short video recommendations by cross-analyzing sequences of both. Furthermore, we have developed a contrastive learning method to oversee the decoupling of user behaviors effectively. Our evaluations demonstrate noticeable improvements through both offline assessments within public datasets and online A/B testing within Kuaishou’s short video recommendation environment on large screen devices. Specifically, our online A/B tests resulted in a 0.55% increase in watch time. These results underscore MODEM’s efficacy in enhancing recommendation quality in shared account contexts.

  Full text in ACM Digital Library

• INDMore to Read at the Los Angeles Times: Solving a Cold Start Problem with LLMs to Improve Story Discovery
  by Franklin Horn (Los Angeles Times), Aurelia Alston (Los Angeles Times) and Won J. You (Los Angeles Times)

  News publishers, who are seeking to grow their digital audience, face a challenge in providing relevant content recommendations for unregistered users arriving directly to article pages. In these cold start scenarios, classic techniques, like asking a user to register and select topics of interest, fall short. We present a contextual targeting approach that leverages the user’s current article choice as an implicit signal of user interests. We designed and developed an interface with recommendations to help users discover more articles. Our A/B testing showed that our models increased click-through rates by 39.4% over a popularity baseline. One of them, a large language model (LLM), generates relevant recommendations that balance immersion and novelty. We discuss the implications of using LLMs for responsibly enhancing user experiences while upholding editorial standards. We identify key opportunities in detecting nuanced user preferences and identifying and interrupting filter bubbles on news publisher sites.

  Full text in ACM Digital Library

• RESMulti-Behavioral Sequential Recommendation
  by Shereen Elsayed (University of Hildesheim), Ahmed Rashed (Volkswagen Financial Services AG) and Lars Schmidt-Thieme (University of Hildesheim)

  Sequential recommendation models are crucial for next-item prediction tasks in various online platforms, yet many focus on a single behavior, neglecting valuable implicit interactions. While multi-behavioral models address this using graph-based approaches, they often fail to capture sequential patterns simultaneously. Our proposed Multi-Behavioral Sequential Recommendation framework (MBSRec) captures the multi-behavior dependencies between the heterogeneous historical interactions via multi-head self-attention. Furthermore, we utilize a weighted binary cross-entropy loss for precise behavior control. Experimental results on four datasets demonstrate MBSRec’s significant outperformance of state-of-the-art approaches. The implementation code is available here.

  Full text in ACM Digital Library

• DEMOMulti-Preview Recommendation via Reinforcement Learning
  by Yang Xu (North Carolina State University), Kuan-Ting Lai (Microsoft), Pengcheng Xiong (Microsoft) and Zhong Wu (Microsoft)

  Preview recommendations serve as a crucial shortcut for attracting users’ attention on various systems, platforms, and webpages, significantly boosting user engagement. However, the variability of preview types and the flexibility of preview duration make it challenging to use an integrated framework for multi-preview recommendations under resource constraints. In this paper, we present an approach that incorporates constrained Q-learning into a notification recommendation system, effectively handling both multi-preview ranking and duration orchestration by targeting long-term user retention. Our method bridges the gap between combinatorial reinforcement learning, which often remains too theoretical for practical use, and segmented modules in production, where model performance is typically compromised due to over-simplification. We demonstrate the superiority of our approach through off-policy evaluation and online A/B testing using Microsoft data.

  Full text in ACM Digital Library

• RESPay Attention to Attention for Sequential Recommendation
  by Yuli Liu (Qinghai University), Min Liu (Qinghai University) and Xiaojing Liu (Qinghai University)

  Transformer-based approaches have demonstrated remarkable success in various sequence-based tasks. However, traditional self-attention models may not sufficiently capture the intricate dependencies within items in sequential recommendation scenarios. This is due to the lack of explicit emphasis on attention weights, which play a critical role in allocating attention and understanding item-to-item correlations. To better exploit the potential of attention weights and improve the capability of sequential recommendation in learning high-order dependencies, we propose a novel sequential recommendation (SR) approach called attention weight refinement (AWRSR). AWRSR enhances the effectiveness of self-attention by additionally paying attention to attention weights, allowing for more refined attention distributions of correlations among items. We conduct comprehensive experiments on multiple real-world datasets, demonstrating that our approach consistently outperforms state-of-the-art SR models. Moreover, we provide a thorough analysis of AWRSR’s effectiveness in capturing higher-level dependencies. These findings suggest that AWRSR offers a promising new direction for enhancing the performance of self-attention architecture in SR tasks, with potential applications in other sequence-based problems as well.

  Full text in ACM Digital Library

• RESPromoting Two-sided Fairness with Adaptive Weights for Providers and Customers in Recommendation
  by Lanling Xu (Renmin University of China), Zihan Lin (KuaiShou Inc.), Jinpeng Wang (Meituan Group), Sheng Chen (Meituan Group), Wayne Xin Zhao (Renmin University of China) and Ji-Rong Wen (Renmin University of China)

  At present, most recommender systems involve two stakeholders, providers and customers. Apart from maximizing the recommendation accuracy, the fairness issue for both sides should also be considered. Most of previous studies try to improve two-sided fairness with post-processing algorithms or fairness-aware loss constraints, which are highly dependent on the heuristic adjustments without respect to the optimization goal of accuracy. In contrast, we propose a novel training framework, adaptive weighting towards two-sided fairness-aware recommendation (named Ada2Fair), which lies in the extension of the accuracy-focused objective to a controllable preference learning loss over the interaction data. Specifically, we adjust the optimization scale of an interaction sample with an adaptive weight generator, and estimate the two-sided fairness-aware weights within model training. During the training process, the recommender is trained with two-sided fairness-aware weights to boost the utility of niche providers and inactive customers in a unified way. Extensive experiments on three public datasets verify the effectiveness of Ada2Fair, which can achieve Pareto efficiency in two-sided fairness-aware recommendation.

  Full text in ACM Digital Library

• DEMORs4rs: Semantically Find Recent Publications from Top Recommendation System-Related Venues
  by Tri Kurniawan Wijaya (Huawei Ireland Research Centre), Edoardo D’Amico (Huawei Ireland Research Centre), Gabor Fodor (Huawei Ireland Research Centre) and Manuel V. Loureiro (Huawei Ireland Research Centre)

  Rs4rs is a web application designed to perform semantic search on recent papers from top conferences and journals related to Recommender Systems. Current scholarly search engine tools like Google Scholar, Semantic Scholar, and ResearchGate often yield broad results that fail to target the most relevant high-quality publications. Moreover, manually visiting individual conference and journal websites is a time-consuming process that primarily supports only syntactic searches. Rs4rs addresses these issues by providing a user-friendly platform where researchers can input their topic of interest and receive a list of recent, relevant papers from top Recommender Systems venues. Utilizing semantic search techniques, Rs4rs ensures that the search results are not only precise and relevant but also comprehensive, capturing papers regardless of
  variations in wording. This tool significantly enhances research efficiency and accuracy, thereby benefitting the research community and public by facilitating access to high-quality, pertinent academic resources in the field of Recommender Systems. Rs4rs is available at https://rs4rs.com.

  Full text in ACM Digital Library

• RESSeCor: Aligning Semantic and Collaborative Representations by Large Language Models for Next-Point-of-Interest Recommendations
  by Shirui Wang (Tongji University), Bohan Xie (Tongji University), Ling Ding (Tongji University), Xiaoying Gao (Tongji University), Jianting Chen (Tongji University) and Yang Xiang (Tongji University)

  The widespread adoption of location-based applications has created a growing demand for point-of-interest (POI) recommendation, which aims to predict a user’s next POI based on their historical check-in data and current location. However, existing methods often struggle to capture the intricate relationships within check-in data. This is largely due to their limitations in representing temporal and spatial information and underutilizing rich semantic features. While large language models (LLMs) offer powerful semantic comprehension to solve them, they are limited by hallucination and the inability to incorporate global collaborative information. To address these issues, we propose a novel method SeCor, which treats POI recommendation as a multi-modal task and integrates semantic and collaborative representations to form an efficient hybrid encoding. SeCor first employs a basic collaborative filtering model to mine interaction features. These embeddings, as one modal information, are fed into LLM to align with semantic representation, leading to efficient hybrid embeddings. To mitigate the hallucination, SeCor recommends based on the hybrid embeddings rather than directly using the LLM’s output text. Extensive experiments on three public real-world datasets show that SeCor outperforms all baselines, achieving improved recommendation performance by effectively integrating collaborative and semantic information through LLMs.

  Full text in ACM Digital Library

• LBRSocial Choice for Heterogeneous Fairness in Recommendation
  by Amanda Aird (University of Colorado Boulder), Elena Štefancová (Comenius University Bratislava), Cassidy All (University of Colorado Boulder), Amy Voida (University of Colorado Boulder), Martin Homola (Comenius University Bratislava), Nicholas Mattei (Tulane University) and Robin Burke (University of Colorado Boulder)

  Algorithmic fairness in recommender systems requires close attention to the needs of a diverse set of stakeholders that may have competing interests. Previous work in this area has often been limited by fixed, single-objective definitions of fairness, built into algorithms or optimization criteria that are applied to a single fairness dimension or, at most, applied identically across dimensions. These narrow conceptualizations limit the ability to adapt fairness-aware solutions to the wide range of stakeholder needs and fairness definitions that arise in practice. Our work approaches recommendation fairness from the standpoint of computational social choice, using a multi-agent framework. In this paper, we explore the properties of different social choice mechanisms and demonstrate the successful integration of multiple, heterogeneous fairness definitions across multiple data sets.

  Full text in ACM Digital Library

• RESThe MovieLens Beliefs Dataset: Collecting Pre-Choice Data for Online Recommender Systems
  by Guy Aridor (Northwestern University), Duarte Goncalves (University College London), Ruoyan Kong (University of Minnesota), Daniel Kluver (University of Minnesota) and Joseph Konstan (University of Minnesota)

  An increasingly important aspect of designing recommender systems involves considering how recommendations will influence consumer choices. This paper addresses this issue by introducing a method for collecting user beliefs about un-experienced goods – a critical predictor of choice behavior. We implemented this method on the MovieLens platform, resulting in a rich dataset that combines user ratings, beliefs, and observed recommendations. We document challenges to such data collection, including selection bias in response and limited coverage of the product space. This unique resource empowers researchers to delve deeper into user behavior and analyze user choices absent recommendations, measure the effectiveness of recommendations, and prototype algorithms that leverage user belief data, ultimately leading to more impactful recommender systems. The dataset can be found at https://grouplens.org/datasets/movielens/ml_belief_2024/.

  Full text in ACM Digital Library

• RESThe Role of Unknown Interactions in Implicit Matrix Factorization — A Probabilistic View
  by Joey De Pauw (University of Antwerp) and Bart Goethals (University of Antwerp)

  Matrix factorization is a well-known and effective methodology for top-k list recommendation. It became widely known during the Netflix challenge in 2006, and since then, many adapted and improved versions have been published. A particularly interesting matrix factorization algorithm called iALS (for implicit Alternating Least Squares) adapts the method for implicit feedback, i.e. a setting where only a very small amount of positive labels are available along with a majority of unknown labels. Compared to the classical task of rating prediction, learning from implicit feedback is applicable to many more domains, as the data is more abundant and requires less effort to elicit from users. However, the sparsity, imbalance, and implicit nature of the signal also pose unique challenges to retrieving the most relevant items to recommend.

  We revisit the role of unknown interactions in implicit matrix factorization. Traditionally, all unknowns are interpreted as negative samples and their importance in the training objective is then down-weighted to balance them out with the known, positive interactions. Interestingly, by adapting a probabilistic view of matrix factorization, we can retain the unknown nature of these interactions by modelling them as either positive or negative. With this new formulation that better fits the underlying data, we gain improved performance on the downstream recommendation task without any computational overhead compared to the popular iALS method.

  This paper outlines the key insights needed to adapt iALS to use logistic regression. Furthermore, a logistic version of the popular full-rank EASE model is introduced in a similar fasion. An extensive experimental evaluation on several real-world datasets demonstrates the effectiveness of our approach. Additionally, a discrepancy between the need for weighting between factorization and autoencoder models is discovered, leading towards a better understanding of these methods.

  Full text in ACM Digital Library

• RESTowards Green Recommender Systems: Investigating the Impact of Data Reduction on Carbon Footprint and Algorithm Performances
  by Giuseppe Spillo (University of Bari Aldo Moro), Allegra De Filippo (DISI Università di Bologna), Cataldo Musto (University of Bari Aldo Moro), Michela Milano (DISI Università di Bologna) and Giovanni Semeraro (University of Bari Aldo Moro)

  This work investigates the path toward green recommender systems by examining the impact of data reduction on both model performance and carbon footprint. In the pursuit of developing energy-efficient recommender systems, we investigated whether and how reducing the training data impacts the performances of several representative recommendation models. In order to obtain a fair comparison, all the models were run based on the implementations available in a popular recommendation library, i.e., RecBole, and used the same experimental settings. Results indicate that: (a) data reduction can be a promising strategy to make recommender systems more sustainable, at the cost of a lower accuracy; (b) training recommender systems with less data makes the suggestions more diverse and less biased. Overall, this study contributes to the ongoing discourse on the development of recommendation models that meet the principles of SDGs, laying the groundwork for the adoption of more sustainable practices in the field.

  Full text in ACM Digital Library

• RESTowards Open-World Recommendation with Knowledge Augmentation from Large Language Models
  by Yunjia Xi (Shanghai Jiao Tong University), Weiwen Liu (Huawei Noah’s Ark Lab), Jianghao Lin (Shanghai Jiao Tong University), Xiaoling Cai (Huawei), Hong Zhu (Huawei), Jieming Zhu (Huawei Noah’s Ark Lab), Bo Chen (Huawei Noah’s Ark Lab), Ruiming Tang (Huawei Noah’s Ark Lab), Weinan Zhang (Shanghai Jiao Tong University) and Yong Yu (Shanghai Jiao Tong University)

  Recommender system plays a vital role in various online services. However, its insulated nature of training and deploying separately within a specific closed domain limits its access to open-world knowledge. Recently, the emergence of large language models (LLMs) has shown promise in bridging this gap by encoding extensive world knowledge and demonstrating reasoning capabilities. Nevertheless, previous attempts to directly use LLMs as recommenders cannot meet the inference latency demand of industrial recommender systems. In this work, we propose an Open-World Knowledge Augmented Recommendation Framework with Large Language Models, dubbed KAR, to acquire two types of external knowledge from LLMs — the reasoning knowledge on user preferences and the factual knowledge on items. We introduce factorization prompting to elicit accurate reasoning on user preferences. The generated reasoning and factual knowledge are effectively transformed and condensed into augmented vectors by a hybrid-expert adaptor in order to be compatible with the recommendation task. The obtained vectors can then be directly used to enhance the performance of any recommendation model. We also ensure efficient inference by preprocessing and prestoring the knowledge from the LLM. Extensive experiments show that KAR significantly outperforms the state-of-the-art baselines and is compatible with a wide range of recommendation algorithms. We deploy KAR to Huawei’s news and music recommendation platforms and gain a 7% and 1.7% improvement in the online A/B test, respectively.

  Full text in ACM Digital Library

• RESΔ-OPE: Off-Policy Estimation with Pairs of Policies
  by Olivier Jeunen (ShareChat) and Aleksei Ustimenko (ShareChat)

  The off-policy paradigm casts recommendation as a counterfactual decision-making task, allowing practitioners to unbiasedly estimate online metrics using offline data. This leads to effective evaluation metrics, as well as learning procedures that directly optimise online success. Nevertheless, the high variance that comes with unbiasedness is typically the crux that complicates practical applications. An important insight is that the difference between policy values can often be estimated with significantly reduced variance, if said policies have positive covariance. This allows us to formulate a pairwise off-policy estimation task: Δ-OPE.

  Δ-OPE subsumes the common use-case of estimating improvements of a learnt policy over a production policy, using data collected by a stochastic logging policy. We introduce Δ-OPE methods based on the widely used Inverse Propensity Scoring estimator and its extensions. Moreover, we characterise a variance-optimal additive control variate that further enhances efficiency. Simulated, offline, and online experiments show that our methods significantly improve performance for both evaluation and learning tasks.

  Full text in ACM Digital Library

Back to program