MicroSSIM: Improved Structural Similarity for Comparing Microscopy Data

Ashesh¹, Joran Deschamps¹ Florian Jug¹

¹Human Technopole, Italy
ECCV 2024

Paper+Supp Code

Summary

We find that the classical SSIM measure is not well suited for comparing predictions made on low-SNR microscopy data with the corresponding high-SNR groundtruth. This kind of comparison is needed for evaluation of unsupervised denoising tasks. We report the phenomenon of saturation, which is the behaviour of SSIM to achieve a high score even when the predictions are far from the groundtruth. We show this unambigously by computing SSIM between a microscopy image and pure noise. We propose a new measure, MicroSSIM, which is an extension of SSIM that is better suited for this domain. We show that MicroSSIM has less saturation and that it focuses on the foreground content. Our approach is such that it can be applied to numerous SSIM variants, and we show this by extending the (Multiscale) MS-SSIM measure to MicroMS3IM.

Failure mode of SSIM on Microscopy Data: (Top) A noisy microscopy image, i.e. a micrograph, its denoised version predicted using N2V, and the corresponding High-SNR (noise free) ground truth is shown. There is a problem in the evaluation of denoising quality, which is that the pixel intensity distribution of the ground truth and the prediction (as shown in respective insets) are very different. This is specifically true for the foreground content which comprises brighter pixels. So applying SSIM directly on it will not give a sensible value. We solve this issue with MicroSSIM. (Bottom) We show one example to demonstrate an apparent counter-intuitive behavior of SSIM. The SSIM between a natural image (taken from Imagenet) and a pure noisy image drawn from the uniform distribution is much lower than the SSIM between a micrograph and a noisy image with identical distribution as before. The expectation naturally is to have SSIM ≈ 0 in both cases. We solve this issue with MicroSSIM and appropriate data pre-processing and show it in bottom right plot where 30 random microscopy images are used.

Our Approach

We remove the background from the ground truth (x) and the prediction (y). We scale the prediction with a scalar. We then compute the SSIM between these background removed images. We use the same scalars for all images in a dataset.

Comparison with Baselines Extreme saturation can be seen in Vanilla SSIM's Structure component (row 1). If we normalize the prediction and ground truth, this does not also solve the issue (row 2) as we observe that background is getting penalized. CARE-SSIM (row 3) applies a linear transformation on the prediction before computing SSIM. It also heavily penalizes the background and therefore is not suitable. MicroSSIM (row 4) is able to focus on the foreground content and has less saturation (darker in color) as compared to vanilla SSIM.

Background removal is important: This is not same as removing mean intensity as removing the mean (row 1) leads to unintuitive contrast component. The foreground and the background are similarly penalized. However, one expects to have more error in the foreground since background usually has low intensity values. Not subtracting out anything (row 2) leads to saturation in the structure component. We show that removing the background (row 3) leads to better results.

BibTeX


        @InProceedings{ashesh_2024_BIC_ECCV,
          author       = {Ashesh, Joran Deschamps and
                          Florian Jug},
          title        = {MicroSSIM: Improved Structural Similarity for Comparing Microscopy Data},
          month        = {October},
          year         = {2024},
          booktitle = {Bio Image Computing workshop at ECCV 2024},
        }