Keynotes

How users evaluate things and each other in social media (PDF)

by Jure Leskovec (Stanford University)

Abstract: Recommender systems are inherently driven by evaluations and reviews provided by the users of these systems. Understanding ways in which users form judgments and produce evaluations can provide insights for modern recommendation systems. Many on-line social applications include mechanisms for users to express evaluations of one another, or of the content they create. In a variety of domains, mechanisms for evaluation allow one user to say whether he or she trusts another user, or likes the content they produced, or wants to confer special levels of authority or responsibility on them. We investigate a number of fundamental ways in which user and item characteristics affect the evaluations in online settings. For example, evaluations are not unidimensional but include multiple aspects that all together contribute to user's overall rating. We investigate methods for modeling attitudes and attributes from online reviews that help us better understand user's individual preferences. We also examine how to create a composite description of evaluations that accurately reflects some type of cumulative opinion of a community. Natural applications of these investigations include predicting the evaluation outcomes based on user characteristics and to estimate the chance of a favorable overall evaluation from a group knowing only the attributes of the group's members, but not their expressed opinions.

Bio: Jure Leskovec is assistant professor of Computer Science at Stanford University where he is a member of the Info Lab and the AI Lab. His research focuses on mining large social and information networks.

Problems he investigates are motivated by large scale data, the Web and on-line media. This research has won several awards including best paper awards at KDD (2005, 2007, 2010), WSDM (2011), ICDM (2011) and ASCE J. of Water Resources Planning and Management (2009), ACM KDD dissertation award (2009), Microsoft Research Faculty Fellowship (2011), Alfred P. Sloan Fellowship (2012) and NSF Early Career Development (CAREER) Award (2011). The research has also been mentioned in popular press outlets such as the New York Times, the Wallstreet Journal, the Washington Post, MIT Technology review, NBC, BBC, CBC and Wired. Jure has authored the Stanford Network Analysis Platform (SNAP), a general purpose network analysis and graph mining library that easily scales to massive networks with hundreds of millions of nodes and billions of edges.

Jure serves on the editorial board of ACM Transactions of the Web and chaired ACM WWW Social networks track (2010, 2011). He received his bachelor's degree in computer science from University of Ljubljana, Slovenia, Ph.D. in machine learning from the Carnegie Mellon University and postdoctoral training from Cornell University. You can follow him on Twitter @jure.

Online Controlled Experiments: Introduction, Learnings, and Humbling Statistics (PDF)

by Ron Kohavi (Microsoft)

Abstract: The web provides an unprecedented opportunity to accelerate innovation by evaluating ideas quickly and accurately using controlled experiments (e.g., A/B tests and their generalizations). Whether for front-end user-interface changes, or backend recommendation systems and relevance algorithms, online controlled experiments are now utilized to make data-driven decisions at Amazon, Microsoft, eBay, Facebook, Google, Yahoo, Zynga, and at many other companies. While the theory of a controlled experiment is simple, and dates back to Sir Ronald A. Fisher's experiments at the Rothamsted Agricultural Experimental Station in England in the 1920s, the deployment and mining of online controlled experiments at scale-thousands of experiments now-has taught us many lessons. We provide an introduction, share real examples, key learnings, cultural challenges, and humbling statistics.

Bio: Ron Kohavi is a partner architect in the Online Services Division. He joined Microsoft in 2005 and founded the Experimentation Platform team in 2006, a "startup" that grew to 50 people. He was previously the director of data mining and personalization at Amazon.com, the Vice President of Business Intelligence at Blue Martini Software (now Escalate). Prior to joining Blue Martini, Kohavi managed the MineSet project, Silicon Graphics' award-winning product for data mining and visualization. He joined Silicon Graphics after getting a Ph.D. in Machine Learning from Stanford University, where he led the MLC++ project, the Machine Learning library in C++ used in MineSet and at Blue Martini Software. Kohavi received his BA from the Technion, Israel. He was the General Chair for KDD 2004. He co-chaired KDD 99's industrial track with Jim Gray and the KDD Cup 2000 with Carla Brodley. He was an invited speaker at the National Academy of Engineering in 2000, a keynote speaker at PAKDD 2001, an invited speaker at KDD 2001's industrial track, and a keynote speaker at EC 10 (2010). He was a member of the editorial board for the Data Mining and Knowledge Discovery journal from its inception for several years and served as a member of the editorial board for the journal of Machine Learning from 1997 to 1999.