CVPR 2021 Tutorial on

Leave Those Nets Alone:
Advances in Self-Supervised Learning

Sunday, June 20 2021, 10:00 - 14:30 EDT (16:00 - 20:30 CET)

The tutorial will be livestreamed on YouTube



Over the last few years, deep learning-based methods have achieved impressive results on image understanding problems. However, real-world vision applications often require models that are able to learn with few or no annotated data. Unfortunately, classic supervised deep learning methods are unable to do that. Furthermore, there is an increased interest for learning image understanding models that can adapt to a wide range of tasks in a data-efficient way, are robust to domain shifts and adversarial perturbations, and are more aligned to how humans see. Such models may require, among other things, a richer training signal than what classic supervised techniques can offer when they rely on image/video-wise category annotations, which is the cheapest and thus the most commonly used type of human annotations. Therefore, one of the next big challenges in computer vision is to devise approaches that would address these important shortcomings.

An important and active research approach for achieving this goal is self-supervised / unsupervised learning. Indeed, the last two years there has been a lot of exciting progress in this area, with many new self-supervised pre-training methods managing to match or even surpass the performance of supervised pre-training. In this tutorial we will provide an in-depth coverage of the various paradigms for self-supervised learning and their recent breakthroughs. Specifically, the tutorial will cover the following subjects (1) contrastive-based self-supervised learning, (2) teacher-student schemes for self-supervised learning, (3) clustering-style self-supervised learning, and (4) multi-modal self-supervised learning.


10:00 - 10:30 EDT (16:00 - 16:30 CET) . Introduction by Andrei Bursuc and Spyros Gidaris [video] [slides]

10:35 - 11:00 PDT (16:35 - 17:00 CET) . Contrastive learning by Aäron van den Oord [video]

11:05 - 12:00 EDT (17:05 - 18:00 CET) . Teacher-student approaches by Spyros Gidaris and Andrei Bursuc [video] [slides]

12:05 - 12:50 EDT (18:05 - 18:50 CET) . Clustering-style self-supervised learning by Mathilde Caron [video] [slides]

12:55 - 13:50 EDT (18:55 - 19:50 CET) . Multi-modal approaches by Jean-Baptiste Alayrac and Adrià Recasens [video] [slides]

13:55 - 14:30 EDT (19:55 - 20:30 CET) . "What is next?" by Andrei Bursuc [video] [slides]

Please contact Spyros Gidaris if you have questions.

This webpage template is by courtesy of Georgia Gkioxari.

Last updated: 24 June 2021