Tutorials

Counterfactual Learning and Evaluation for Recommender Systems: Foundations, Implementations, and Recent Advances

by Yuta Saito (Cornell University, USA) and Thorsten Joachims (Cornell University, USA)

Counterfactual estimators enable the use of existing log data to estimate how some new target recommendation policy would have performed, if it had been used instead of the policy that logged the data. We say that those estimators work “off-policy”, since the policy that logged the data is different from the target policy. In this way, counterfactual estimators enable Off-policy Evaluation (OPE) akin to an unbiased offline A/B test, as well as learning new recommendation policies through Off-policy Learning (OPL). The goal of this tutorial is to summarize Foundations, Implementations, and Recent Advances of OPE/OPL. Specifically, we will introduce the fundamentals of OPE/OPL and provide theoretical and empirical comparisons of conventional methods. Then, we will cover emerging practical challenges such as how to take into account combinatorial actions, distributional shift, fairness of exposure, and two-sided market structures. We will then present Open Bandit Pipeline, an open-source package for OPE/OPL, and how it can be used for both research and practical purposes. We will conclude the tutorial by presenting real-world case studies and future directions.

The learning outcomes of this tutorial are to enable the participants (such as applied researchers, practitioners, and students):

• to know fundamental concepts and conventional methods of OPE/OPL
• to be familiar with recent advances to address practical challenges such as fairness of exposure
• to understand how to implement OPE/OPL in their research and applications
• to be aware of remaining challenges and opportunities in the area

This tutorial is aimed at an audience with intermediate experience in machine learning, information retrieval, or recommender systems who are interested in using OPE/OPL methods in their research and applications. Participants are expected to have basic knowledge of machine learning, probability theory, and statistics. The tutorial will provide practical examples based on Python code and Jupyter Notebooks.

Multi-Modal Recommender Systems: Hands-On Exploration

by Quoc-Tuan Truong (Singapore Management University, Singapore), Aghiles Salah (Rakuten Institute of Technology, France), and Hady W. Lauw (Singapore Management University, Singapore)

Recommender systems typically learn from user-item preference data such as ratings and clicks. This information is sparse in nature, i.e., observed user-item preferences often represent less than 5% of possible interactions. One promising direction to alleviate data sparsity is to leverage auxiliary information that may encode additional clues on how users consume items. Examples of such data (referred to as modalities) are social networks, item’s descriptive text, product images. The objective of this tutorial is to offer a comprehensive review of recent advances to represent, transform and incorporate the different modalities into recommendation models. Moreover, through practical hands-on sessions, we consider cross model/modality comparisons to investigate the importance of different methods and modalities. The hands-on will be conducted with Cornac (https://cornac.preferred.ai), a comparative framework for multimodal recommender systems.

The tutorial is structured as follows. After providing an overview of important families of preference models, we discuss how multi-modal recommender systems may be devised in combination with models that represent specific modalities (auxiliary data beyond user-item preferences). Subsequently, we dive into each of the three main modalities of interest, namely: text, image, and graph, while identifying relevant algorithms under each modality. This is followed by an investigation of cross-modal utilization, including which modality one should rely on, whether a model designed for one modality can work with another, and which model to use for a given modality.

This tutorial is meant for both practitioners seeking applicable experience, as well researchers interested in recent and future research directions in multimodal recommender systems. Attendees are expected to have basic knowledge of Python and machine learning. While familiarity with recommender systems is a plus, it is not a must. Please bring your own laptop for this hands-on session.

End-to-End Session-Based Recommendation on GPU

by Gabriel de Souza Pereira Moreira (NVIDIA, Brazil), Sara Rabhi (NVIDIA, Canada), Ronay Ak (NVIDIA, USA), and Benedikt Schifferer (NVIDIA, USA)

Session-based recommendation, a sub-area of sequential recommendation, has been an important task in online services like e-commerce and news portals, where most users either browse in a session anonymously or may have very distinct interests for different sessions. Session-Based Recommender Systems (SBRS) have been proposed to model the sequence of interactions within the current user session, where a session is a short sequence of user interactions typically bounded by user inactivity. They have recently gained popularity due to their ability to capture short-term or contextual user preferences towards items.

The usage of RNN-based architectures for SBRS, which started with the seminal work of GRU4Rec, was proposed to learn more complex sequential patterns on user sessions. One of the advantages of neural networks (NNs) is their flexibility to include additional features from item metadata and user context, improving the recommendation accuracy. In recent works, state-of-the-art NLP approaches have inspired RecSys practitioners and researchers to leverage the self-attention mechanism and the transformer-based architectures for sequential and session-based recommendation, and these models achieved promising results.

In this regard, the aim of this tutorial is to present to the participants: (i) an introduction on the main concepts and algorithms for session-based recommendation, (ii) how to build, train and evaluate a session-based recommendation model based on RNN and Transformer architectures, and (iii) how to speed up with GPUs the entire RecSys pipeline which encompasses feature engineering, preprocessing, training, evaluation and inference using NVIDIA Merlin- an open source ecosystem for large-scale deep learning recommender systems.

The tutorial is designed as a combination of theoretical lectures and practical exercises based on Python code and PyTorch framework. The participants are required to have basic knowledge of recommender systems, deep learning, Python and PyTorch framework. NVIDIA will host the tutorial on its computing infrastructure, providing the environment, dataset, example Jupyter notebooks and the GPU. Therefore, participants will be able to easily join the tutorial via their web browsers, and are only expected to bring their own laptops and have an internet connection.

Prepare For Your Hands-on Tutorial Training

To get the most from your hands-on learning experience, please complete these steps prior to getting started:

1. Create or log into your NVIDIA Developer Program account. This account will provide you with access to all of the training materials during the tutorial.
2. Visit websocketstest.courses.nvidia.com and make sure all three test steps are checked “Yes.” This will test the ability for your system to access and deliver the training contents. If you encounter issues, try updating your browser. Note: Only Chrome and Firefox are supported.
3. Check your bandwidth. 1 Mbps downstream is required and 5 Mbps is recommended. This will ensure consistent streaming of audio/video during the tutorial to avoid glitches and delays.

Pursuing Privacy in Recommender Systems: the View of Users and Researchers from Regulations to Applications

by Vito Walter Anelli (Polytechnic University of Bari, Italy), Luca Belli (Twitter, USA), Yashar Deldjoo, Tommaso Di Noia, Antonio Ferrara, Fedelucio Narducci, and Claudio Pomo (Polytechnic University of Bari, Italy)

To provide users with accurate and tailored recommendations, mainstream recommender systems handle past preferences about items, behavioral and demographic data, context, and social connections. However, gathering this data may pose privacy threats to users, thus spurring data security laws and regulations. Nevertheless, in contrast to other AI communities, the topic of privacy has been quite under-researched in the community of recommender systems, making it critical for designers and service providers to understand how to create reliable recommendations while still protecting their customers’ privacy.

This tutorial aims to bridge the gap in perspectives and advances between the RecSys and other AI communities and increase privacy awareness in RSs. The tutorial will cover the following topics:

• The Data Paradox: Privacy and Utility in the Era of Regulations
• Foundations to Privacy-Preserving RSs: Differential Privacy and Cryptographic Methods
• Learning Paradigms for Privacy-Oriented Recommender Systems (with a focus on privacy-by-design architectures like federated learning)
• Threats to privacy-oriented RSs
• Trending Research and Open Challenges

We target an intermediate audience of researchers and practitioners willing to delve into the privacy aspects of recommender systems. The tutorial will conclude with a practical session and a plenary discussion of the future of privacy in recommender systems.

Conversational Recommendation: Formulation, Methods, and Evaluation

by Wenqiang Lei (National University of Singapore, Singapore), Chongming Gao (University of Science and Technology of China, China), and Maarten de Rijke (University of Amsterdam & Ahold Delhaize, Netherlands)

Recommender systems have demonstrated great success in information seeking. However, traditional recommender systems work in a static way, estimating user preferences on items from past interaction history. This prevents recommender systems from capturing dynamic and fine-grained preferences of users. Conversational recommender systems bring a revolution to existing recommender systems. They are able to communicate with users through natural language, which enables them to explicitly elicit user preferences by asking whether a user likes an attribute or item or not. Based on information shared through users’ responses, a recommender system can produce more accurate and personalized recommendations.

We identify five emerging trends in the general area of conversational recommender systems: (1) Question-based user preference elicitation; (2) Multi-turn conversational recommendation strategies; (3) Dialogue understanding and generation; (4) Exploitation-exploration trade-offs; and (5) Evaluation and user simulation. This tutorial covers these five directions, providing a review of existing approaches and progress on each topic.

By presenting the emerging and promising topic of conversational recommender systems, we aim to provide take-aways to practitioners to build their own systems. We also want to stimulate more ideas and discussions with audiences on core problems of this topic such as task formalization, dataset collection, algorithm development, and evaluation, with the ambition of facilitating the development of conversational recommender systems. This tutorial was previously presented at SIGIR’ 20, and this time we improve the content.

Bias Issues and Solutions in Recommender System

by Jiawei Chen (University of Science and Technology of China, China), Xiang Wang (National University of Singapore, Singapore), Fuli Feng (National University of Singapore, Singapore), and Xiangnan He (University of Science and Technology of China, China)

Recommender systems (RS) have demonstrated great success in information seeking. Recent years have witnessed a large number of work on inventing recommendation models to better fit user behavior data. However, user behavior data is observational rather than experimental. This makes various biases widely exist in the data, including but not limited to selection bias, position bias, exposure bias. Blindly fitting the data without considering the inherent biases will result in many serious issues, e.g., the discrepancy between offline evaluation and online metrics, hurting user satisfaction and trust on the recommendation service, etc. To transform the large volume of research models into practical improvements, it is highly urgent to explore the impacts of the biases and develop debiasing strategies when necessary. Therefore, bias issues and solutions in recommender systems have drawn great attention from both academic and industry.

In this tutorial, we aim to provide an systemic review of existing work on this topic. We will introduce seven types of biases in recommender system, along with their definitions and characteristics; review existing debiasing solutions, along with their strengths and weaknesses; and identify some open challenges and future directions. We hope this tutorial could stimulate more ideas on this topic and facilitate the development of debiasing recommender systems. This tutorial was previously presented at The Web Conference 2021.