At Shopify, we serve product recommendations to customers across millions of merchants’ online stores. It is a challenge to provide optimized recommendations to all of these independent merchants; one model might create an overall improvement in our metrics on aggregate, but significantly degrade recommendations for some stores. To ensure we provide high quality recommendations to all merchant segments, we develop several models that work best in different situations based on offline evaluation. Learning which strategy best works for a given segment also allows us to start off new stores with good recommendations, without necessarily needing to rely on an individual store amassing large amounts of traffic. In production, the system will start out with the best strategy for a given merchant, and then adjust to the current environment using multi-armed bandits. Collectively, this methodology allows us to optimize the types of recommendations served on each store.

Full text in ACM Digital Library

Recommendation systems are of crucial importance for a variety of modern apps and web services, such as news feeds, social networks, e-commerce, search, etc. To achieve peak prediction accuracy, modern recommendation models combine deep learning with terabyte-scale embedding tables to obtain a fine-grained representation of the underlying data. Traditional inference serving architectures require deploying the whole model to standalone servers, which is infeasible at such massive scale.
In this paper, we provide insights into the intriguing and challenging inference domain of online recommendation systems. We propose the HugeCTR Hierarchical Parameter Server (HPS), an industry-leading distributed recommendation inference framework, that combines a high-performance GPU embedding cache with an hierarchical storage architecture, to realize low-latency retrieval of embeddings for online model inference tasks. Among other things, our HPS features (1) a redundant hierarchical storage system, (2) a novel high-bandwidth cache to accelerate parallel embedding lookup on NVIDIA GPUs, (3) online training support and (4) light-weight APIs for integration into existing large-scale recommendation workflows. To demonstrate the capabilities of HPS, we conducted extensive studies by using both synthetically engineered and public datasets. We show that HPS can dramatically reduce the end-to-end inference latency, achieving 5~62x speedup (depending on the batch size) for popular recommendation models over CPU baseline implementations. Through multi-GPU concurrent deployment, HPS can greatly increase the inference QPS.

Full text in ACM Digital Library

Peloton has a subscription-based service offering access to a rich catalog of high-quality fitness classes. Not only is there a wide diversity amidst these classes but this inventory is constantly changing. This dynamic inventory introduces a new challenge for our recommender systems – surfacing timely content in addition to relevant content. We are often faced with a set of classes that are filmed and timely only during a narrow window, for example, holiday-themed classes. During this time, they need to reach a sizable audience to satisfy business goals, while also preserving user engagement. However, naïvely surfacing timely content has the potential to hurt user engagement goals. We have to factor in individual interests when choosing who to display this timely content. Our recommender system, which is already aware of users’ interests, is best placed to balance relevance and timeliness.
In this talk, we will show how we have created a system and algorithms for artificially increasing the impressions of selected sets of classes, which we call boosting. We open up control over our recommender systems for our marketing and production partners, who are able to entered timed boosts for selected classes. We show how these boosts are then honored by both our batch and real-time recommendation engines to selectively display the boosted classes higher in rankings so as to get them more visibility. We will discuss a naïve boosting algorithm, which produced high lifts in impressions but at the cost of user engagement. We will then demonstrate a batch optimization approach that leads to a better balance between engagement and impressions. We will look at results of several A/B tests on our user base, and end with a summary of benefits of this system as well as emerging challenges.

Full text in ACM Digital Library