Advanced Topics in ML:
Self-Supervised Learning for Imaging

About

Credits: 2
Prerequisites: Linear algebra, optimization, and probability (or equivalent background) — no prior exposure to self-supervised learning assumed
Instructor: Ashesh — AI Department, IIT Hyderabad
Lecture venue: TBD

Course Overview

Most of machine learning’s recent progress — large language models, foundation models, generative image and video systems — is built on a deceptively simple idea: a model can learn a great deal directly from raw, unlabelled data, without a human ever supplying ground truth. This course is a focused, technical treatment of that idea — self-supervised learning (SSL) — with imaging as the running domain.

The course opens by asking why self-supervision has become central to contemporary ML, and surveys the architectural building blocks (convolutional networks, transformers, diffusion models, flow matching) that make it possible. It then goes deep on three functional pillars of SSL:

Self-supervised restoration — denoising, inpainting and super-resolution.
Representation learning — contrastive, non-contrastive, and masked-pretraining methods, and how they underlie today’s vision foundation models.
Test-time finetuning and domain adaptation — adapting a pretrained model to a new data distribution with minimal supervision.

The course is project-based: students work through a single project spanning all three pillars on a small, real, free-tier-compute-friendly dataset, and present papers throughout the term.

Course Content

Module 1 — Why Self-Supervision Is Central

The shift from task-specific supervised models to pretrained foundation models
Building blocks: convolutional architectures, transformer architecture, diffusion models, flow matching
Why the label-scarcity gap is quite extreme in scientific and medical imaging

Module 2 — Self-Supervised Restoration

Inverse problem formalism: forward operators, noise models, classical regularization
Self-supervised denoising I: Noise2Noise and the corruption-averaging family
Self-supervised denoising II: Noise2Void / blind-spot networks, SURE and UNSURE
Inpainting and incomplete operators, including atom probe tomography reconstruction as a case study
Paper presentations

Module 3 — Representation Learning

Contrastive and non-contrastive self-supervised pretraining (SimCLR-style, BYOL-style, DINO-style)
Masked-image pretraining and foundation models for segmentation (MAE-style, SAM, DINOv2)
Paper presentations

Module 4 — Test-Time Finetuning and Domain Adaptation

Test-time training, adapters, and low-rank finetuning (LoRA)
Adapting restoration and representation models to a shifted data distribution
Paper presentations

Module 5 — Course Project

Single project spanning all three pillars on a small public dataset.
Phase 1: self-supervised restoration on one imaging modality
Phase 2: representation pretraining and evaluation on the same data
Phase 3: test-time finetuning to a second, distinct modality
Checkpoints and final project presentations

Schedule

Approximate — exact session count per topic will be adjusted based on pacing.

Sessions	Module	Format
1–3	Why self-supervision is central; building blocks (CNNs, transformers, diffusion, flow matching)	Lecture
4–11	Self-supervised restoration (denoising theory, decomposition, inpainting/APT)	Lecture + coding lab + paper presentation
12–16	Representation learning (contrastive/non-contrastive/masked, foundation models)	Lecture + coding lab + paper presentation
17–21	Test-time finetuning / domain adaptation	Lecture + coding lab + paper presentation
22–28	Course project: design, implementation, checkpoints, presentations	Project work + presentations

Evaluation Criteria

Course project (three-phase SSL pipeline): 40%
Assignments, Paper reading, and Presentations: 20%
Class participation: 10%
Mid-sem exam : 10-15%
End-sem exam : 15-20%

Generative AI (Gen-AI) Policy

The use of AI tools for understanding papers, writing code, and related tasks is encouraged; however, such use must be explicitly disclosed. If AI has been used, one is expected to be able to explain the generated output, for example, the purpose of a function or the rationale for using a decorator in some function. In other words, the evaluation will change depending upon whether, and to what extent, AI was used in assignments and the course project.

References

Tachella, J. & Davies, M. Self-Supervised Learning from Noisy and Incomplete Data — review covering the restoration module in depth, with an accompanying tutorial (slides, videos, Colab demos) at tachella.github.io
Balestriero, Ibrahim, Sobal et al. (Meta AI), “A Cookbook of Self-Supervised Learning” (2023) — a long, book-style arXiv document covering SSL broadly (pretext tasks, architectures, training recipes, evaluation), written explicitly to be a practical reference rather than a paper.
Bednarski, M. & Roith, T. Introduction to Regularization and Learning Methods for Inverse Problems — lecture notes for the classical inverse-problem background (arXiv:2508.18178)

Advanced Topics in ML: Self-Supervised Learning for Imaging