Schedule
The Summer School will contain lectures and exercises mainly covered by the four invited speakers. Local speakers will provide examples of domain adaptation in their research and relate the topic to the research at DIKU and DTU Informatics. In addition to the scientific program there will be a Summer School dinner Thursday evening.
A tentative program is available as pdf here.
Invited Lecturers
Corinna Cortes
The lectures will cover:
- Variations of support vector machines (SVMs) for domain adaptation.
- Boosting methods for domain adaptation.
Suggested reading material:
- Lorenzo Bruzzone and Mattia Marconcini: Domain Adaptation Problems: A DASVM Classification Technique and a Circular Validation Strategy, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 32, No. 5, May 2010.
- Yishay Mansour, Mariano Schain: Robust Domain Adaptation. Proceedings of the International Symposium on Artificial Intelligence and Mathematics, 2012.
- Paul Viola, Michael Jones: Robust Real-time Object Detection. International Journal of Computer Vision 57, no. 2: 137-154, 2001.
- Amaury Habrard, Jean-Philippe Peyrache, Marc Sebban: Domain Adaptation with Good Edit Similarities: a Sparse Way to deal with Scaling and Rotation Problems in Image Classification. ICTAI 2011. 181-188, 2011.
Mehryar Mohri
Lecture 1: Convex Optimization.
This lecture covers:
- convex optimization (definition, properties, theorems).
- linear programming (LP).
- quadratic programming (QP).
- semidefinite programming (SDP).
Suggested reading material:
- Convex Optimization. Boyd, Stephen P. and Vandenberghe, Lieven. Cambridge University Press, 2004.
- Foundations of Machine Learning. Mohri, Rostami, and Talwalkar, MIT Press, 2012. (Note that every attendant of the Summer School receives a copy of this book.)
Lecture 2: Adaptation in Regression: Generalization Bounds and Algorithms.
This lecture covers:
- learning bounds for regularization-based algorithm.
- discrepancy minimization algorithm and optimization.
- empirical results.
Suggested reading material:
- Yishay Mansour, Mehryar Mohri, Afshin Rostamizadeh: Domain Adaptation: Learning Bounds and Algorithms. COLT 2009.
- Corinna Cortes, Mehryar Mohri: Domain Adaptation in Regression. ALT 2011: 308-323.
- Mehryar Mohri, Andres Munoz Medina: New Analysis and Algorithm for Learning with Drifting Distributions. arXiv:1205.4343, 2012.
Yishay Mansour
Lecture 1: Generalization Bounds
This lecture covers:
- Review of generalization bound (VC dimension, Radamacher complexity)
- Single domain source adaptation bounds: discrepancy distance, bounded VC dimension bounds. Re-weighing source examples.
Suggested reading material:
- Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, Jennifer Wortman Vaughan: A theory of learning from different domains. Machine Learning 79(1-2): 151-175, 2010.
- Yishay Mansour, Mehryar Mohri and Afshin Rostamizadeh: Domain Adaptation: Learning Bounds and Algorithms. COLT 2009.
Lecture 2: Multi-source Adaptation
The motivation is based on the observation that in many cases you have huge amounts of unlabeled data from the target domain, and fairly good classifiers from related domain. The goal is to combine those classifiers to predict in the new target domain, for which we have only unlabeled examples. The surprising result is that there exists a class of combination rules which guarantees, in certain aspects, to transfer the predictability power of the source domains to the new target domain.
Suggested reading material:
- Yishay Mansour, Mehryar Mohri and Afshin Rostamizadeh: Multiple Source Adaptation and the Renyi Divergence. UAI 2009.
- Yishay Mansour, Mehryar Mohri and Afshin Rostamizadeh: Domain Adaptation with Multiple Sources. NIPS 2008.
Trevor Darrell
Lectures: Domain Adaptation in Object Recognition
Domain adaptation is an important emerging topic in computer vision. I'll review recent studies of domain shift in the context of object recognition, and cover new methods that adapt object models acquired in a particular visual domain to new imaging conditions by learning a transformation that minimizes the effect of domain-induced changes in the feature distribution. Transformations may be learned in a supervised manner and can be applied to categories for which there are no labeled examples in the new domain. While I'll focus the presentation on object recognition tasks, the transform-based adaptation technique is general and could be applied to non-image data. I'll also describe methods for multi-source domain adaptation, and adaptation in the context of discriminative learning frameworks, as well as available online datasets relevant to vision-based domain adaptation.
Tutorial: Domain Adaptation in Object Recognition
In this practical exercise, we will explore domain adaptation problems and standard algorithms on the PASCAL motorbike->bike (Ref. 1) and the Office dataset (Ref. 2). The exercise is organized by Trevor Darrell and held by Kersten Petersen.
Suggested reading material:
- Yusuf Aytar and Andrew Zisserman: Tabula Rasa: Model Transfer for Object Category Detection. CVPR 2011.
- Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell: Adapting Visual Category Models to New Domains. ECCV 2010.
- Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell: What You Saw is Not What You Get: Domain Adaptation Using Asymmetric Kernel Transforms. CVPR 2011.
Guest Lecturers
Tobias Glasmachers
Lecture: Multi-class Support Vector Machines
Extending the standard Support Vector Machine classifier to problems with more
than two classes is a non-trivial issue, for which a multitude of solutions
has been proposed. The lecture will be a round trip through the diverse domain
of large-margin multi-category classification. We will see how most approaches
can be understood systematically within a unifying framework.
Suggested reading material:
The audience should be familiar with "standard" support vector machines for binary classification as presented in the introductory lectures.
Tutorial: Machine Learning with the Shark
The "Shark" machine learning library is a modular C++ library for the design and optimization of adaptive systems. In this hands-on practical course on machine learning with C++ we will go through the different steps of the machine learning processing chain: importing data, pre-processing, training a model, optimizing parameters, and testing the final performance.
Important: It is highly recommended that participants have the Shark library, Version 3, pre-installed (and example programs tested) on their laptop computers. Installing shark requires nothing but a modern C++ compiler and a recent version of the boost libraries. The Shark library is ready for download from
Suggested reading material:
- Christian Igel, Verena Heidrich-Meisner, and Tobias Glasmachers: Shark. Journal of Machine Learning Research 9, pp. 993-996, 2008
Marco Loog
Covariate Shift : Some Theory, Some Examples, and Some Observations
Suggested reading material:
Local Lecturers
Christian Igel
Lecture 1: Supervised Machine Learning
Machine learning is about developing and analyzing algorithms that automatically improve with experience. Such algorithms are already an integral part of today's computing systems, for example in search engines, recommender systems, or biometrical applications. This short lecture will provide a gentle introduction to supervised machine learning.
Suggested reading material:
Lecture 2: An Introduction to Markov Random Fields and Restricted Boltzmann Machines
The tutorial considers Markov random fields (i.e., undirected graphical models) with a special focus on restricted Boltzmann machines (RBMs). In the first part, undirected graphical models are introduced, where image denoising serves as an application example. The second part concentrates on RBMs, which are undirected graphical models describing stochastic neural networks. Recently, RBMs have raised attention as building block of deep belief networks (DBNs).
Marleen de Bruijne
Domain adaptation in medical imaging
This talk will cover current approaches to coping with common variations in medical imaging data as well as a proof-of-concept study of transfer learning for medical image segmentation.
