Session 2: Click-Through Rate prediction

Date: Wednesday September 20, 11:15 AM – 12:35 PM (GMT+8)
Room: Hall 406D
Session Chair: Zhenhua Dong
Parallel with: Session 1: Collaborative filtering 1

• RESGradient Matching for Categorical Data Distillation in CTR Prediction
  by Cheng Wang (School of Cyber Science and Engineering,Huazhong University of Science and Technology, Wuhan), Jiacheng Sun (Huawei Noah’s Ark Lab), Zhenhua Dong (Huawei Noah’s Ark Lab), Ruixuan Li (School of Computer Science and Technology,Huazhong University of Science and Technology, Wuhan) and Rui Zhang (ruizhang.info).

  The cost of hardware and energy consumption on training a click-through rate (CTR) model is highly prohibitive. A recent promising direction for reducing such costs is data distillation with gradient matching, which aims to synthesize a small distilled dataset to guide the model to a similar parameter space as those trained on real data. However, there are two main challenges to implementing such a method in the recommendation field: (1) The categorical recommended data are high dimensional and sparse one- or multi-hot data which will block the gradient flow, causing backpropagation-based data distillation invalid. (2) The data distillation process with gradient matching is computationally expensive due to the bi-level optimization. To this end, we investigate efficient data distillation tailored for recommendation data with plenty of side information where we formulate the discrete data to the dense and continuous data format. Then, we further introduce a one-step gradient matching scheme, which performs gradient matching for only a single step to overcome the inefficient training process. The overall proposed method is called Categorical data distillation with Gradient Matching (CGM), which is capable of distilling a large dataset into a small of informative synthetic data for training CTR models from scratch. Experimental results show that our proposed method not only outperforms the state-of-the-art coreset selection and data distillation methods but also has remarkable cross-architecture performance. Moreover, we explore the application of CGM on continual updating and mitigate the effect of different random seeds on the training results.

  Full text in ACM Digital Library

• RESDeep Situation-Aware Interaction Network for Click-Through Rate Prediction
  by Yimin Lv (Institute of Software, Chinese Academy of Sciences), Shuli Wang (Meituan), Beihong Jin (Institute of Software, Chinese Academy of Sciences), Yisong Yu (Institute of Software, Chinese Academy of Sciences), Yapeng Zhang (Meituan), Jian Dong (Meituan), Yongkang Wang (Meituan), Xingxing Wang (Meituan) and Dong Wang (Meituan).

  User behavior sequence modeling plays a significant role in Click-Through Rate (CTR) prediction on e-commerce platforms. Except for the interacted items, user behaviors contain rich interaction information, such as the behavior type, time, location, etc. However, so far, the information related to user behaviors has not yet been fully exploited. In the paper, we propose the concept of a situation and situational features for distinguishing interaction behaviors and then design a CTR model named Deep Situation-Aware Interaction Network (DSAIN). DSAIN first adopts the reparameterization trick to reduce noise in the original user behavior sequences. Then it learns the embeddings of situational features by feature embedding parameterization and tri-directional correlation fusion. Finally, it obtains the embedding of behavior sequence via heterogeneous situation aggregation. We conduct extensive offline experiments on three real-world datasets. Experimental results demonstrate the superiority of the proposed DSAIN model. More importantly, DSAIN has increased the CTR by 2.70\%, the CPM by 2.62\%, and the GMV by 2.16\% in the online A/B test. Now, DSAIN has been deployed on the Meituan food delivery platform and serves the main traffic of the Meituan takeout app. Our source code is available at https://github.com/W-void/DSAIN

  Full text in ACM Digital Library

• RESAutoOpt: Automatic Hyperparameter Scheduling and Optimization for Deep Click-through Rate Prediction
  by Yujun Li (Noah’s Ark Lab), Xing Tang (Noah’s Ark Lab), Bo Chen (Noah’s Ark Lab), Yimin Huang (Noah’s Ark Lab), Ruiming Tang (Noah’s Ark Lab) and Zhenguo Li (Noah’s Ark Lab).

  Click-through Rate (CTR) prediction is essential for commercial recommender systems. Recently, to improve the prediction accuracy, plenty of deep learning-based CTR models have been proposed, which are sensitive to hyperparameters and difficult to optimize well. General hyperparameter optimization methods fix these hyperparameters across the entire model training and repeat them multiple times. This trial-and-error process not only leads to suboptimal performance but also requires non-trivial computation efforts. In this paper, we propose an automatic hyperparameters scheduling and optimization method for deep CTR models, \emph{AutoOpt}, making the optimization process more stable and efficient. Specifically, the whole training regime is firstly divided into several consecutive stages, where a data-efficient model is learned to model the relation between model states and prediction performance. To optimize the stage-wise hyperparameters, AutoOpt uses the \textit{global} and \textit{local} scheduling modules to propose proper hyperparameters for the next stage based on the training in the current stage. Extensive experiments on three public benchmarks are conducted to validate the effectiveness of AutoOpt. Moreover, AutoOpt has been deployed onto an advertising platform and a music platform, where online A/B tests also demonstrate superior improvement.

  Full text in ACM Digital Library

• INDLoss Harmonizing for Multi-Scenario CTR Prediction
  by Congcong Liu (JD.com), Liang Shi (JD.com), Pei Wang (JD.com), Fei Teng (JD.com), Xue Jiang (JD.com), Changping Peng (JD.com), Zhangang Lin (JD.com) and Jingping Shao (JD.com).

  Large-scale industrial systems often include multiple scenarios to satisfy diverse user needs. The common approach of using one model per scenario does not scale well and not suitable for minor scenarios with limited samples. An solution is to train a model on all scenarios, which can introduce domination and bias from the main scenario. MMoE-like structures have been proposed for multi-scenario prediction, but they do not explicitly address the issue of gradient unbalancing. This work proposes an adaptive loss harmonizing (ALH) algorithm for multi-scenario CTR prediction. It balances training by dynamically adjusting the learning speed, resulting in improved prediction performance. Experiments conducted on real production dataset and a rigorous A/B test prove the superiority of our method.

  Full text in ACM Digital Library

Back to program