Session 8: Sequential Recommendation 1

Date: Wednesday October 16, 10:25 AM – 11:05 AM (GMT+2)
Room: Petruzzelli Theater
Session Chair: Domonkos Tikk

• RES 🕓5CALRec: Contrastive Alignment of Generative LLMs For Sequential Recommendation
  by Yaoyiran Li (University of Cambridge), Xiang Zhai (Google), Moustafa Alzantot (Google Research), Keyi Yu (Google), Ivan Vulić (University of Cambridge), Anna Korhonen (University of Cambridge) and Mohamed Hammad (Google)

  Traditional recommender systems such as matrix factorization methods have primarily focused on learning a shared dense embedding space to represent both items and user preferences. Subsequently, sequence models such as RNN, GRUs, and, recently, Transformers have emerged and excelled in the task of sequential recommendation. This task requires understanding the sequential structure present in users’ historical interactions to predict the next item they may like. Building upon the success of Large Language Models (LLMs) in a variety of tasks, researchers have recently explored using LLMs that are pretrained on vast corpora of text for sequential recommendation. To use LLMs for sequential recommendation, both the history of user interactions and the model’s prediction of the next item are expressed in text form. We propose CALRec, a two-stage LLM finetuning framework that finetunes a pretrained LLM in a two-tower fashion using a mixture of two contrastive losses and a language modeling loss: the LLM is first finetuned on a data mixture from multiple domains followed by another round of target domain finetuning. Our model significantly outperforms many state-of-the-art baselines (+37% in Recall@1 and +24% in NDCG@10) and our systematic ablation studies reveal that (i) both stages of finetuning are crucial, and, when combined, we achieve improved performance, and (ii) contrastive alignment is effective among the target domains explored in our experiments.

  Full text in ACM Digital Library

• RES 🕓5A Pre-trained Zero-shot Sequential Recommendation Framework via Popularity Dynamics
  by Junting Wang (Urbana-Champaign), Praneet Rathi (Urbana-Champaign) and Hari Sundaram (Urbana-Champaign)

  This paper proposes a novel pre-trained framework for zero-shot cross-domain sequential recommendation without auxiliary information. While using auxiliary information (e.g., item descriptions) seems promising for cross-domain transfer, a cross-domain adaptation of sequential recommenders can be challenging when the target domain differs from the source domain—item descriptions are in different languages; metadata modalities (e.g., audio, image, and text) differ across source and target domains. If we can learn universal item representations independent of the domain type (e.g., groceries, movies), we can achieve zero-shot cross-domain transfer without auxiliary information. Our critical insight is that user interaction sequences highlight shifting user preferences via the popularity dynamics of interacted items. We present a pre-trained sequential recommendation framework: PrepRec, which utilizes a novel popularity dynamics-aware transformer architecture. Through extensive experiments on five real-world datasets, we show that PrepRec, without any auxiliary information, can zero-shot adapt to new application domains and achieve competitive performance compared to state-of-the-art sequential recommender models. In addition, we show that PrepRec complements existing sequential recommenders. With a simple post-hoc interpolation, PrepRec improves the performance of existing sequential recommenders on average by 11.8% in Recall@10 and 22% in NDCG@10. We provide an anonymized implementation of PrepRec at https://github.com/CrowdDynamicsLab/preprec.

  Full text in ACM Digital Library

• RES 🕓5Scaling Law of Large Sequential Recommendation Models
  by Gaowei Zhang (Renmin University of China), Yupeng Hou (University of California San Diego), Hongyu Lu (Tencent), Yu Chen (Tencent), Wayne Xin Zhao (Renmin University of China) and Ji-Rong Wen (Renmin University of China)

  Scaling of neural networks has recently shown great potential to improve the model capacity in various fields. Specifically, model performance has a power-law relationship with model size or data size, which provides important guidance for the development of large-scale models. However, there is still limited understanding on the scaling effect of user behavior models in recommender systems, where the unique data characteristics (e.g., data scarcity and sparsity) pose new challenges in recommendation tasks.

  In this work, we focus on investigating the scaling laws in large sequential recommendation models. Specifically, we consider a pure ID-based task formulation, where the interaction history of a user is formatted as a chronological sequence of item IDs. We don’t incorporate any side information (e.g., item text), to delve into the scaling law’s applicability from the perspective of user behavior. We successfully scale up the model size to 0.8B parameters, making it feasible to explore the scaling effect in a diverse range of model sizes. As the major findings, we empirically show that the scaling law still holds for these trained models, even in data-constrained scenarios. We then fit the curve for scaling law, and successfully predict the test loss of the two largest tested model scales.

  Furthermore, we examine the performance advantage of scaling effect on five challenging recommendation tasks, considering the unique issues (e.g., cold start, robustness, long-term preference) in recommender systems. We find that scaling up the model size can greatly boost the performance on these challenging tasks, which again verifies the benefits of large recommendation models.

  Full text in ACM Digital Library

• REPR 🕓10ReChorus2.0: A Modular and Task-Flexible Recommendation Library
  by Jiayu Li (Tsinghua University), Hanyu Li (Tsinghua University), Zhiyu He (Tsinghua University), Weizhi Ma (Tsinghua University), Peijie Sun (Tsinghua University), Min Zhang (Tsinghua University) and Shaoping Ma (Tsinghua University)

  With the applications of recommendation systems rapidly expanding, an increasing number of studies have focused on every aspect of recommender systems with different data inputs, models, and task settings. Therefore, a flexible library is needed to help researchers implement the experimental strategies they require. Existing open libraries for recommendation scenarios have enabled reproducing various recommendation methods and provided standard implementations. However, these libraries often impose certain restrictions on data and seldom support the same model to perform different tasks and input formats, limiting users from customized explorations. To fill the gap, we propose ReChorus2.0, a modular and task-flexible library for recommendation researchers. Based on ReChorus, we upgrade the supported input formats, models, and training&evaluation strategies to help realize more recommendation tasks with more data types. The main contributions of ReChorus2.0 include: (1) Realization of complex and practical tasks, including re-ranking and CTR prediction tasks; (2) Inclusion of various context-aware and re-ranking recommenders; (3) Extension of existing and new models to support different tasks with the same models; (4) Support of highly-customized input with impression logs, negative items, or click labels, as well as user, item, and situation contexts. To summarize, ReChorus2.0 serves as a comprehensive and flexible library that better addresses the practical problems in the recommendation scenario and caters to more diverse research needs. The implementation and detailed tutorials of ReChorus2.0 can be found at https://github.com/THUwangcy/ReChorus.

  Full text in ACM Digital Library

• RES 🕓15Dynamic Stage-aware User Interest Learning for Heterogeneous Sequential Recommendation
  by Weixin Li (Shenzhen University), Xiaolin Lin (Shenzhen University), Weike Pan (Shenzhen University) and Zhong Ming (Shenzhen Technology University)

  Sequential recommendation has been widely used to predict users’ potential preferences by learning their dynamic user interests, for which most previous methods focus on capturing item-level dependencies. Despite the great success, they often overlook the stage-level interest dependencies. In real-world scenarios, user interests tend to be staged, e.g., following an item purchase, a user’s interests may undergo a transition into the subsequent phase. And there are intricate dependencies across different stages. Meanwhile, users’ behaviors are usually heterogeneous, including auxiliary behaviors (e.g., examinations) and target behaviors (e.g., purchases), which imply more fine-grained user interests. However, existing methods have limitations in explicitly modeling the relationships between the different types of behaviors. To address the above issues, we propose a novel framework, i.e., dynamic stage-aware user interest learning (DSUIL), for heterogeneous sequential recommendation, which is the first solution to model user interests in a cross-stage manner. Specifically, our DSUIL consists of four modules: (1) a dynamic graph construction module transforms a heterogeneous sequence into several subgraphs to model user interests in a stage-wise manner; (2) a dynamic graph convolution module dynamically learns item representations in each subgraph; (3) a behavior-aware subgraph representation learning module learns the heterogeneous dependencies between behaviors and aggregates item representations to represent the staged user interests; and (4) an interest evolving pattern extractor learns the users’ overall interests for the item prediction. Extensive experimental results on two public datasets show that our DSUIL performs significantly better than the state-of-the-art methods.

  Full text in ACM Digital Library

Back to program