College students are required to select a major but are often provided with only a modest amount of support in making this important decision. A poor decision is detrimental to the student, since it may result in the student later switching to a different major with a delay in graduation—or even result in the student leaving the university. This also impacts the university since time to graduation and retention rate are used to evaluate the quality of a university. There is a general lack of research on recommender systems for college majors, with the most relevant systems focusing on course-level recommendations. This study describes and evaluates a recommender system for selecting an undergraduate major, utilizing nine years of historical student data from a large university. The system bases its recommendations on the courses that the student takes in the first few years of college, and how well they performed in these courses. The system is designed to recommend majors that the student is likely to be interested in and will perform well in. Recommendations are evaluated based on the likelihood that the student’s actual major was in the top five recommended majors, and whether the student performed above average in that major. The recommendation system dramatically outperforms the baseline strategy of randomly selecting a major, and when the recommendation is followed the student is 12% more likely to perform above average in the major.

Full text in ACM Digital Library

Recent work has shown that, despite their simplicity, item-based models optimised through ridge regression can attain highly competitive results on collaborative filtering tasks. As these models are analytically computable and thus forgo the need for often expensive iterative optimisation procedures, they are an attractive choice for practitioners. We study the applicability of such closed-form models to implicit-feedback collaborative filtering when additional side-information or metadata about items is available. Two complementary extensions to the easer paradigm are proposed, based on collective and additive models. Through an extensive empirical analysis on several large-scale datasets, we show that our methods can effectively exploit side-information whilst retaining a closed-form solution, and improve upon the state-of-the-art without increasing the computational complexity of the original easer approach. Additionally, empirical results demonstrate that the use of side-information leads to more “long tail” items being recommended, benefiting the recommendations’ coverage of the item catalogue.

Full text in ACM Digital Library

Researchers and service providers have focused on leveraging social information acquired from interactions between users to improve the accuracy of system recommendations. However, few have explained the characteristics of music recommendations through interpersonal relationships. To investigate how interpersonal relationships affect users’ evaluation of music recommendation, we conducted a survey-based study that compared two types of recommendation channels—interpersonal (i.e., from friends) and non-interpersonal (i.e., from systems). We found that relevance was evaluated higher in music recommended from non-interpersonal channels on average, while diversity, novelty, and serendipity were higher in interpersonal channels. Non-interpersonal channels surpassed interpersonal channels in terms of convenience, frequency, and adoption rate. These results illustrate that interpersonal and non-interpersonal channels have different strengths and that digital streaming platforms, which have mainly provided system recommendations thus far, need to better support interpersonal channels for richer user experience.

Full text in ACM Digital Library

Frameworks that aid the development of Recommender Systems (RSs) are extremely important, since they reduce their development cost by offering reusable tools, as well as implementations of common strategies and popular models. However, it is still hard to find a framework that also provides full abstraction over data set conversion, support for deep learning-based approaches, extensible models and reproducible evaluations. This work introduces a new framework that not only provides several modules to avoid repetitive development work, but also to assist practitioners with these existing challenges. Our evaluation procedure ensures that RSs developed using this new approach are consistent and extensible, by analysing their predictive performance and certain characteristics of their implementation.

Full text in ACM Digital Library

With over 20,000 tracks being released each day, recommendation systems that power music streaming services should not only be responsive to such large volumes of content, but also be adept at understanding the impact of such new releases on, both, users’ listening behavior and popularity of artists. Inferring the causal impact of new track releases is critical to fully characterizing the interplay between artists and listeners, as well as among the artists. In this study, we infer and quantify causality using a diffusion-regression state-space model that constructs counterfactual outcomes using a set of synthetic controls, which predict potential outcomes in absence of the intervention. Based on large scale experiments spanning over 21 million users and 1 billion streams on a real world streaming platform, our findings suggest that releasing a new track has a positive impact on the popularity of other tracks by the same artist. Interestingly, other related and competing artists also benefit from a new track release, which hints at the presence of a positive platform-effect wherein some artists gain significantly from activities of other artists.

Full text in ACM Digital Library

Music streaming is inherently sequential in nature, with track sequence information playing a key role in user satisfaction with recommended music. In this work, we investigate the role audio characteristics of music content play in understanding music streaming sessions. Focusing on 18 audio attributes (e.g. dancability, acousticness, energy), we formulate audio transitioning in a session as a multiple changepoint detection problem, and extract latent states of different audio attributes within each session. Based on insights from large scale music streaming data from a popular music streaming platform, we investigate questions around the extent to which audio characteristics fluctuate within streaming sessions, the heterogeneity across different audio attributes and their impact on user satisfaction. Furthermore, we demonstrate the promise of such audio-based characterizing of sessions in better sequencing tracks in a session, and highlight the potential gains in user satisfaction on offer. We discuss implications on the design of track sequencing models, and identify important prediction tasks to further research on the topic.

Full text in ACM Digital Library

Multi-language recipe personalisation and recommendation is an under-explored field of information retrieval in academic and production systems. The existing gaps in our current understanding are numerous, even on fundamental questions such as whether consistent and high-quality recipe recommendation can be delivered across languages. Motivated by this need, we consider the multi-language recipe recommendation setting and present grounding results that will help to establish the potential and absolute value of future work in this area. Our work draws on several billion events from millions of recipes, with published recipes and users incorporating several languages, including Arabic, English, Indonesian, Russian, and Spanish. We represent recipes using a combination of normalised ingredients, standardised skills and image embeddings obtained without human intervention. In modelling, we take a classical approach based on optimising an embedded bi-linear user-item metric space towards the interactions that most strongly elicit cooking intent. For users without interaction histories, a bespoke content-based cold-start model that predicts context and recipe affinity is introduced. We show that our approach to personalisation is stable and scales well to new languages. A robust cross-validation campaign is employed and consistently rejects baseline models and representations, strongly favouring those we propose. Our results are presented in a language-oriented (as opposed to model-oriented) fashion to emphasise the language-based goals of this work. We believe that this is the first large-scale work that evaluates the value and potential of multi-language recipe recommendation and personalisation.

Full text in ACM Digital Library

Targeting system is an essential part of computational advertising. It allows advertisers to select and reach their targeted users. Due to various advertising goals and the demand for making budget plans, advertisers have a strong will to configure the final targeting results, or they can become very cautious in spending money on advertising campaigns. Meanwhile, to guarantee the advertising performance, the targeted users should also be relevant to the ads of the advertisers. Recent targeting methods are mainly based on tags produced by the Data Management Platform (DMP) which is easy for the advertisers to configure the targeting results. However, in such methods, the relevance between the targeted users and ads is not technically evaluated and cannot be guaranteed. The biggest challenge is that it is hard for a machine learning model to both model the relevance and take account of the advertiser’s configuration demands. In this paper, we propose a novel relevance-driven and configurable targeting framework called Smart Targeting to solve the problem. Specifically, different from Tag-wise Targeting, we first use a relevance model to retrieve the most relevant users for the ads. To further enable the advertisers to configure the final results, we develop a Delay Intervention Mechanism to leverage the power of DMP. As far as we know, this is the first attempt of combining relevance modeling and advertiser intervention into a unified targeting system. We implement and evaluate our framework on JD.com platform with over 300 million users and the results show that it can bring significant improvements to the core indicators such as CTR and eCPM. The long term monitoring also demonstrates that Smart Targeting gradually becomes the most popular targeting tool after its release.

Full text in ACM Digital Library

Word2vec is a powerful machine learning tool that emerged from Natural Language Processing (NLP) and is now applied in multiple domains, including recommender systems, forecasting, and network analysis. As Word2vec is often used off the shelf, we address the question of whether the default hyperparameters are suitable for recommender systems. The answer is emphatically no. In this paper, we first elucidate the importance of hyperparameter optimization and show that unconstrained optimization yields an average 221% improvement in hit rate over the default parameters. However, unconstrained optimization leads to hyperparameter settings that are very expensive and not feasible for large scale recommendation tasks. To this end, we demonstrate 138% average improvement in hit rate with a runtime budget-constrained hyperparameter optimization. Furthermore, to make hyperparameter optimization applicable for large scale recommendation problems where the target dataset is too large to search over, we investigate generalizing hyperparameters settings from samples. We show that applying constrained hyperparameter optimization using only a 10% sample of the data still yields a 91% average improvement in hit rate over the default parameters when applied to the full datasets. Finally, we apply hyperparameters learned using our method of constrained optimization on a sample to the Who To Follow recommendation service at Twitter and are able to increase follow rates by 15%.

Full text in ACM Digital Library