Rethinking Collaborative Filtering: A Practical Perspective on State-Of-The-Art Research Based on Real-World Insights and Challenges
by Noam Koenigstein (Microsoft)
A decade has passed since the seminal Netflix Prize competition and Collaborative Filtering (CF) models are still at the forefront of Recommender System research. Significant progress has been achieved over this time, yet key aspects of the basic problem formulation have not been seriously challenged. Most state-of-the-art models still assume a supervised model in which the ultimate goal is to predict future user-item interactions based on the generalization of historical data.
We wish to initiate a discussion on some key assumptions behind much of the mainstream research: What is the difference between predicting future user actions and optimizing Key Performance Indicators (KPIs)? Does a trade-off between accuracy and diversity really exist? Is supervised CF based on historical data still relevant in the age of modern reinforcement learning models such as contextual bandits? What evaluation metrics can be used prior to online experimentations and what are their limitations? Our main thesis is that at the core of all these issues lies a gap between most Collaborative Filtering models and the true objective of industry systems.
About the Speaker
Noam Koenigstein received the B.Sc. degree in computer science (cum laude) from the Technion – Israel Institute of Technology, Haifa, Israel, in 2007 and the M.Sc. degree in electrical engineering from Tel-Aviv University, Tel-Aviv, Israel, in 2009. In 2013 he received a Ph.D. degree from the School of Electrical Engineering, Tel-Aviv University. In 2011, he joined the Xbox Machine Learning research team in Microsoft, where he developed the algorithm for Xbox recommendations serving millions of users worldwide. Since 2014 he manages the recommendations research team for Microsoft’s Store. His research interests include Machine Learning, Information Retrieval, and large-scale Data-Mining, with specific focus on Recommender Systems.
Slides
https://recsys.acm.org/…/RecSys-17-Industry-Presentation-Microsoft.pdf
Recommendation Applications and Systems at Electronic Arts
by John Kolen (EA)
The digital game industry has recently adopted recommendation systems to provide suitable game and content choices to players. Recommendations in digital games have several unique applications and challenges compared to movies and books. Designers must adopt different architectures and algorithms to overcome these challenges. In this talk, we describe the game recommendation system at Electronic Arts. It leverages heterogeneous player data across many games to provide intelligent recommendations. We discuss three example applications: recommending games for purchase, suitable game map, and game difficulty.
We developed one flexible recommendation system to satisfy the need of different applications and that executes data-driven algorithms such collaborative filtering and multi-armed bandit. The centralized system leverages entire player and game data for all recommendation applications in digital games, supports unified roll-out and update, and at the same time measures the performance together via A/B testing experiments. Moreover, the one system strategy is easy to generate consistent recommendations across multiple games and platforms. We tested these recommendation applications in EA website and games, and observed significant improvements in click-through-rate and engagement.
About the Speaker
Dr. Kolen’s interests include artificial intelligence, distributed systems, neural networks and cognitive science. His career has threaded through academia, government, and industry. He taught computer science at the University of West Florida. At the Institute for Human and Machine Cognition, he secured and worked on several governmental research contracts. His industry
experience spans small start ups, such as BSecure Technologies and Reputation.com, to large companies–Electronic Arts (EA) and Google. Currently, he leads the Intelligent Systems group at EA that helps game studios find ways to leverage AI and data driven solutions within their games. His
applied research topics have included identifying Martian geological formations (for Spirit and Opportunity rovers), modeling ground-based laser marksmen, and computed tomography of debris fields from holograms during armor penetration tests.
Dr. Kolen has authored seven refereed journal publications, two edited books, six book chapters, and twenty-four refereed conference papers. His research topics include neural networks, musical beat tracking, cognitive science, and algorithms. He holds three US patents, with another six currently applied for. Dr. Kolen received his Masters and Ph.D. in computer science from The Ohio
State University. His B.A. in computer science is from the University of California, San Diego.
Slides
https://recsys.acm.org/…/RecSys-17-Industry-Presentation-EA.pdf
Search Ranking And Personalization at AirBnB
by Mihajlo Grbovic (AirBnB)
Search ranking is a fundamental problem of crucial interest to major Internet companies, including web search engines, content publishing websites and marketplaces. However, despite sharing some common characteristics a one-size-fits-all solution does not exist in this space. Given a large difference in content that needs to be ranked and the parties affected by ranking, each search ranking problem is somewhat specific. Correspondingly, search ranking at Airbnb is quite unique, being a two-sided marketplace in which one needs to optimize for host and guest preferences, in a world where a user rarely consumes the same item twice and one listing can accept only one guest for a certain set of dates. In this talk, I will discuss challenges we have encountered and Machine Learning solutions we have developed for listing ranking at Airbnb. Specifically, the listing ranking problem boils down to prioritizing listings that are appealing to the guest but at the same time demoting listings that would likely reject the guest, which is not easily solvable using basic matrix completion or a straightforward linear model. I will shed the light on how we jointly optimize the two objectives by leveraging listing quality, location relevance, reviews, host response time as well as guest and host preferences and past booking history. Finally, we will talk about our recent work on using neural network models to train listing and query embeddings for purposes of enhancing search personalization, broad search and type-ahead suggestions, which are core concepts in any modern search.
About the Speaker
Mihajlo Grbovic, Ph.D. is a Senior Machine Learning Scientist on the Search Ranking Team at Airbnb. Prior to that, he was a Senior Research Manager at Yahoo Labs working on Advertising Sciences. He has more than 10 years of technical experience in applied Machine Learning, acting as a Science Lead in a portfolio of advertising technology projects on Monetization of Tumblr, Yahoo Email and Yahoo Search. Some of his biggest accomplishments include building a large scale Interest and Gender Targeting Pipeline for Tumblr, training Email Classifiers used in Yahoo Mail Smart Views that millions of people interact with every day, and introducing the next generation query-ad matching algorithm to Yahoo Sponsored Search. Dr. Grbovic published more than 40 peer-reviewed publications at top Machine Learning and Web Science Conferences and co-authored more than 10 pending patents. His work was featured in Wall Street Journal, Scientific American, MIT Technology Review, Popular Science and Market Watch.
Slides
https://astro.temple.edu/~tua95067/Mihajlo_RecSys2017.pptx
Bootstrapping a Destination Recommender System
by Neal Lathia (Skyscanner)
Years ago, Skyscanner started it’s “everywhere” search, allowing users to find the cheapest countries to travel to. This feature evolved into an `inspiration feed;’ a stream of destinations, again ordered by price. However, price is just one of many factors that can make a place attractive. In this talk, I’ll discuss how we’ve bootstrapped a destination recommender system to augment Skyscanner’s destination feeds with wisdom-of-the-crowd recommendations, and give an overview of experiments that gauge how localised and personalised recommendations affects user engagement in different parts of the Android and iOS apps.
There are a variety of challenges that we had to tackle in this domain, ranging from data sourcing, sampling, and segmenting, to metric and algorithm selection, and building a pipeline that could facilitate rapid online and offline experimentation. We now have a system that uses the rich implicit data generated by Skyscanner’s millions of users alongside a set of diverse algorithmic approaches to compute destination recommendations. Experimental features that use this pipeline are also collecting unique interaction data that is being analysed to further personalise users’ recommendations. This talk will give an overview of the journey so far and some potential future directions and research challenges for recommendation in the travel domain.
About the Speaker
Dr Neal Lathia is a Senior Data Scientist in Skyscanner’s London office, where he is working with engineering teams to design and build machine learning features in the Skyscanner apps. Prior to that, he was a Senior Research Associate in the Computer Laboratory, University of Cambridge and a postdoctoral researcher at University College London. Neal has a PhD in recommender systems from the Department of Computer Science, University College London.
Slides
https://www.slideshare.net/neal.lathia/bootstrapping-a-destination-recommendation-engine
