CRCS Lunch Seminar

Date: Wednesday, October 30, 2013
Time: 12:00pm – 1:30pm
Place: Maxwell Dworkin 119

Speaker: Jasper Snoek, Harvard CRCS

Title: Bayesian Optimization for Machine Learning and Science

Subtitle:  Developing Bayesian optimization as a methodology to make machine learning more accessible and effective and as a general optimization tool for science

Abstract:

Recent advances in machine learning are starting to have a profound impact throughout the sciences and industry.  However, many of the most powerful machine learning models remain challenging to use effectively by all but a select few domain experts.  How can we make machine learning more accessible to non-experts?  A major hindrance is that the use of machine learning algorithms frequently involves careful tuning of various meta-parameters such as learning parameters and model hyperparameters. Unfortunately, this tuning is often a ``black art'' requiring expert experience, rules of thumb, or sometimes brute-force search. There is therefore great appeal for automatic approaches that can optimize the performance of any given learning algorithm to the problem at hand.  We develop a principled approach to this problem through constructing a statistical model of the functional mapping between these parameters and a given objective which we can iteratively refine and query.  The resulting "Bayesian optimization" procedure was able to find better parameters for multiple recent machine learning models than the experts who developed them and achieved state of the art results on various benchmark problems.  Naturally, many more general problems across the sciences involve a similar form of iterative parameter tuning and recent work has been focused on developing a general tool for tuning parameters for arbitrary problems.

In this talk I will give an overview of this approach, detail some applications to problems in rehabilitation science and assistive technology, and discuss some exciting new applications with collaborators at Harvard and MIT.  Time permitting, I'll give a quick tutorial of the open-source code package we have developed to perform Bayesian optimization.

Bio:

Jasper Snoek is a CRCS postdoctoral fellow at Harvard University and a member of the Harvard Intelligent and Probabilistic Systems Group. His research interests are primarily in machine learning with an emphasis on Bayesian statistics.  He completed a PhD in machine learning at the University of Toronto and has been involved in numerous projects that involve the application of machine learning to problems in health informatics and assistive technology. He was formerly a postdoctoral fellow in the machine learning group at the University of Toronto and a member of the Intelligent Assistive Technology and Systems Lab.

Photo(s) from the Event: