Over the last few years, deep learning-based methods have achieved impressive results on image understanding problems. However, real-world vision applications often require models that are able to learn with few or no annotated data. Unfortunately, classic supervised deep learning methods are unable to do that. Furthermore, there is an increased interest for learning image understanding models that can adapt to a wide range of tasks in a data-efficient way, are robust to domain shifts and adversarial perturbations, and are more aligned to how humans see. Such models may require, among other things, a richer training signal than what classic supervised techniques can offer when they rely on image/video-wise category annotations, which is the cheapest and thus the most commonly used type of human annotations. Therefore, one of the next big challenges in computer vision is to devise approaches that would address these important shortcomings.

An important and active research approach for achieving this goal is self-supervised / unsupervised learning. Indeed, the last two years there has been a lot of exciting progress in this area, with many new self-supervised pre-training methods managing to match or even surpass the performance of supervised pre-training. In this tutorial we will provide an in-depth coverage of the various paradigms for self-supervised learning and their recent breakthroughs. Specifically, the tutorial will cover the following subjects (1) contrastive-based self-supervised learning, (2) teacher-student schemes for self-supervised learning, (3) clustering-style self-supervised learning, and (4) multi-modal self-supervised learning.