【denoise】图像降噪专题
- 一文道尽传统图像降噪方法 link
《Image Denoising with Deep Convolutional Neural Networks》链接
《Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising》Dncnn
《Noise2Noise: Learning Image Restoration without Clean Data》Noise2Noise
《Semantic Image Inpainting with Deep Generative Models》链接
《Image De-raining Using a Conditional Generative Adversarial Network》链接
《DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks》DeblurGAN
课题：如何解决基于生成对抗网络的去噪任务的领域失配问题？
一、整体思路
1.1 提出问题：如何针对真实的图像噪声，设计出有效的图像去噪网络(包含网络框架、损失函数及训练策略)来解决领域失配问题，是真实图像去噪领域另一关键问题。
1.2 解决办法：
- 通过对图像上存在的复杂噪声进行建模，可以准确刻画并模拟生成真实图像噪声，从而为后续去噪网络提供大规模可靠的训练数据对。
- 针对传统图像去噪网络存在的领域失配问题，设计并实现适应于真实图像去噪任务的网络框架以及相应的损失函数和训练策略，从而实现具有较强泛化能力的高性能真实图像去噪。
二、研究内容
2.1 图像去噪网络架构的研究
2.2 图像去噪网络的训练策略研究
2.3 图像去噪网络中的损失函数设计研究
三、数据集:
本课题拟采取三种类型的数据对来增强真实图像去噪的泛化能力。这三种类型的数据对分别为：
3.1 真实的含噪图像与其对应的干净图像
3.2 基于条件生成对抗网络生成的含噪图像以及干净图像
3.3 原始噪声图像以及基于条件生成对抗网络生成的含噪图像，需要指出的是，这里输入𝐲与输出𝐱 + 𝐧是干净图像𝐱的不同含噪图像。
四、Tips
4.1 三种类型的数据量比例作为超参数进行调整优化。
4.2 损失函数采用 SSIM Loss + L1 Loss
4.3 网络结构：U-Net、ResNet、DenseNet
【转】Awesome Image or Video Denoising Algorithms
转自github (https://hub.fastgit.org/z-bingo/awesome-image-denoising-state-of-the-art)
图像/视频去噪算法资源集锦
【导读】图像去噪是指减少数字图像中噪声的过程。随着深度学习的发展，也有许多深度学习方法被用于图像/视频去噪。本文整理了一些去噪算法与数据集。
Collection of popular and reproducible image denoising works.
I will update the document when I access the new work for image or video denoising. Everyone could pull requests or remind me to update if you access the latest work.
This collection is based on the summary of wenbihan's work.
Contents
- Denoising Algorithms
1.1 Filter
1.2 Sparse Coding
1.3 Effective Prior
1.4 Low Rank
1.5 Deep Learning
1.6 Sparsity and Low-rankness Combined
1.7 Combined with High-Level Tasks
1.8 Image Noise Level Estimation - Benchmark and Dataset
2.1 Novel Benchmark
2.2 Commonly Used Denoising Dataset - Others
3.1 Commonly Used Image Quality Metric Code
Denoising Algorithms
Filter
- NLM [Web] [Code] [PDF]
- A non-local algorithm for image denoising (CVPR 05), Buades et al.
- Image denoising based on non-local means filter and its method noise thresholding (SIVP2013), B. Kumar
- BM3D [Web] [Code] [PDF]
- Image restoration by sparse 3D transform-domain collaborative filtering (SPIE Electronic Imaging 2008), Dabov et al.
- PID [Web] [Code] [PDF]
- Progressive Image Denoising (TIP 2014), C. Knaus et al.
Sparse Coding
- KSVD [Web] [Code] [PDF]
- Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries (TIP 2006), Elad et al.
- LSSC [Web] [Code] [PDF]
- Non-local Sparse Models for Image Restoration (ICCV 2009), Mairal et al.
- NCSR [Web] [Code] [PDF]
- Nonlocally Centralized Sparse Representation for Image Restoration (TIP 2012), Dong et al.
- OCTOBOS [Web] [Code] [PDF]
- Structured Overcomplete Sparsifying Transform Learning with Convergence Guarantees and Applications (IJCV 2015), Wen et al.
- GSR [Web] [Code] [PDF]
- Group-based Sparse Representation for Image Restoration (TIP 2014), Zhang et al.
- TWSC [Web] [Code] [PDF]
- A Trilateral Weighted Sparse Coding Scheme for Real-World Image Denoising (ECCV 2018), Xu et al.
Effective Prior
- EPLL [Web] [Code] [PDF]
- From Learning Models of Natural Image Patches to Whole Image Restoration (ICCV2011), Zoran et al.
- GHP [[Web]][Code] [PDF]
- Texture Enhanced Image Denoising via Gradient Histogram Preservation (CVPR2013), Zuo et al.
- PGPD [[Web]][Code] [PDF]
- Patch Group Based Nonlocal Self-Similarity Prior Learning for Image Denoising (ICCV 2015), Xu et al.
- PCLR [[Web]][Code] [PDF]
- External Patch Prior Guided Internal Clustering for Image Denoising (ICCV 2015), Chen et al.
Low Rank
- SAIST [Web] [Code by request] [PDF]
- Nonlocal image restoration with bilateral variance estimation: a low-rank approach (TIP2013), Dong et al.
- WNNM [Web] [Code] [PDF]
- Weighted Nuclear Norm Minimization with Application to Image Denoising (CVPR2014), Gu et al.
- Multi-channel WNNM [Web] [Code] [PDF]
- Multi-channel Weighted Nuclear Norm Minimization for Real Color Image Denoising (ICCV 2017), Xu et al.
Deep Learning
-
- Shrinkage Fields for Effective Image Restoration (CVPR 2014), Schmidt et al.
-
- Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration (TPAMI 2016), Chen et al.
-
- Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections (NIPS2016), Mao et al.
-
- Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising (TIP2017), Zhang et al.
-
- MemNet: A Persistent Memory Network for Image Restoration (ICCV2017), Tai et al.
-
- Learning Pixel-Distribution Prior with Wider Convolution for Image Denoising (Arxiv), Liu et al.
-
- L_p-Norm Constrained Coding With Frank-Wolfe Network (Arxiv), Sun et al.
-
- Non-Local Color Image Denoising with Convolutional Neural Networks (CVPR 2017), Lefkimmiatis.
-
Deep image prior [Web] [Code] [PDF]
- Deep Image Prior (CVPR 2018), Ulyanov et al.
-
- xUnit: Learning a Spatial Activation Function for Efficient Image Restoration (Arxiv), Kligvasser et al.
-
- Universal Denoising Networks : A Novel CNN Architecture for Image Denoising (CVPR 2018), Stamatios Lefkimmiatis.
-
Wavelet-CNN [Web] [Code] [PDF]
- Multi-level Wavelet-CNN for Image Restoration (Arxiv), Liu et al.
-
- FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising (TIP), Zhang et al.
-
- Fully Convolutional Pixel Adaptive Image Denoiser (Arxiv), Cha et al.
-
- Toward Convolutional Blind Denoising of Real Photographs (Arxiv), Guo et al.
-
Noise2Noise [Web] [TF Code] [Keras Unofficial Code] [PDF]
- Noise2Noise: Learning Image Restoration without Clean Data (ICML 2018), Lehtinen et al.
-
- Universal Denoising Networks- A Novel CNN Architecture for Image Denoising (CVPR 2018), Lefkimmiatis.
-
- Neural Nearest Neighbors Networks (NIPS 2018), Plotz et al.
-
- Non-Local Recurrent Network for Image Restoration (NIPS 2018), Liu et al.
-
- Burst Denoising with Kernel Prediction Networks (CVPR 2018), Ben et al.
-
- Multi-Kernel Prediction Networks for Denoising of Burst Images (ArXiv 2019), Marinc et al.
-
- Deep Burst Denoising (ArXiv 2017), Clement et al.
- End-to-End Denoising of Dark Burst Images Using Recurrent Fully Convolutional Networks (ArXiv 2019), Zhao et al.
-
- Image denoising and restoration with CNN-LSTM Encoder Decoder with Direct Attention (ArXiv 2018), Haque et al.
-
- GRDN: Grouped Residual Dense Network for Real Image Denoising and GAN-based Real-world Noise Modeling (CVPR 2019), Kim et al.
-
Deformable KPN [Web] [Code] [PDF]
- Learning Deformable Kernels for Image and Video Denoising (ArXiv 2019), Xu et al.
-
BayerUnify BayerAug [Web] [Code] [PDF]
- Learning Raw Image Denoising With Bayer Pattern Unification and Bayer Preserving Augmentation (CVPR 2019), Liu et al.
-
- A Deep Motion Deblurring Network Based on Per-Pixel Adaptive Kernels With Residual Down-Up and Up-Down Modules (CVPR 2019), Sim et al.
-
- Real Image Denoising with Feature Attention (ArXiv 2019), Anwar et al.
-
- EDVR: Video Restoration With Enhanced Deformable Convolutional Networks (CVPR 2019), Wang et al.
-
- DVDnet: A Fast Network for Deep Video Denoising (ArXiv 2019), Tassano et al.
-
FastDVDNet [Web] [Code] [An Unofficial PyTorch Code] [PDF]
- FastDVDnet: Towards Real-Time Video Denoising Without Explicit Motion Estimation (ArXiv 2019), Tassano et al.
-
- ViDeNN: Deep Blind Video Denoising (ArXiv 2019), Calus et al.
-
Multi-Level Wavelet-CNN [Web] [Code] [PDF]
- Multi-Level Wavelet Convolutional Neural Networks (IEEE Access), Liu et al.
-
- Pyramid Read Image Denoising Network (Arxiv 2019), Zhao et al.
-
- CycleISP: Real Image Restoration via Improved Data Synthesis (CVPR 2020), Zamir et al.
-
- MIRNEt: Learning Enriched Features for Real Image Restoration and Enhancement (ECCV 2020), Zamir et al.
Sparsity and Low-rankness Combined
- STROLLR-2D [PDF] [Code]
- When Sparsity Meets Low-Rankness: Transform Learning With Non-Local Low-Rank Constraint for Image Restoration (ICASSP 2017), Wen et al.
Combined with High-Level Tasks
- Meets High-level Tasks [PDF] [Code]
- When Image Denoising Meets High-Level Vision Tasks: A Deep Learning Approach (IJCAI 2018), Liu et al.
Image Noise Level Estimation
- SINLE [PDF] [Code] [Slides]
- Single-image Noise Level Estimation for Blind Denoising (TIP 2014), Liu et al.
- CBDNet [Code] [PDF]
- Toward Convolutional Blind Denoising of Real Photographs (Arxiv), Guo et al.
Benchmark and Dataset
Novel Benchmark
- ReNOIR [Web] [Data] [PDF]
- RENOIR - A Dataset for Real Low-Light Image Noise Reduction (Arxiv 2014), Anaya, Barbu.
- PolyU [Web] [Data] [PDF]
- Real-world Noisy Image Denoising: A New Benchmark (Arxiv), Xu et al.
- Nam [Web] [PDF]
- A Holistic Approach to Cross-Channel Image Noise Modeling and its Application to Image Denoising (CVPR 2016), Nam et al.
- Darmstadt (DND) [Web] [Data] [PDF]
- Benchmarking Denoising Algorithms with Real Photographs (CVPR 2017), Plotz et al.
- SIDD [Web]
- A High-Quality Denoising Dataset for Smartphone Cameras.
Commonly Used Denoising Dataset
Others
Commonly Used Image Quality Metric Code
-
PSNR (Peak Signal-to-Noise Ratio) [Wiki] [Matlab Code] [Python Code] [PyTorch Code]
-
SSIM (Structural similarity) [Wiki] [Matlab Code] [Python Code] [PyTorch Code]
-
NIQE (Naturalness Image Quality Evaluator) [Web] [Matlab Code] [Python Code]
去噪论文合集
2019年state-of-the-art
| Model | Published | Code | Title |
|---|---|---|---|
| GRDN | CVPR2019 | Code | GRDN: Grouped Residual Dense Network for Real Image Denoising and GAN-based Real-world Noise Modeling |
| RFCN | arxiv | Code/Web | End-to-End Denoising of Dark Burst Images Using Recurrent Fully Convolutional Networks |
| Deformable KPN | arxiv | Code | Learning Deformable Kernels for Image and Video Denoising |
| BayerUnify BayerAug | CVPR2019 | Code | Learning Raw Image Denoising With Bayer Pattern Unification and Bayer Preserving Augmentation |
| RDU-UD | CVPR2019 | Code | A Deep Motion Deblurring Network Based on Per-Pixel Adaptive Kernels With Residual Down-Up and Up-Down Modules |
| RIDNet | ICCV2019 | Code | Real Image Denoising with Feature Attention |
| PRIDNet | VCIP2019 | Code | Pyramid Real Image Denoising Network |
| RNAN | ICLR2019 | Code | Residual Non-local Attention Networks for Image Restoration |
| VDN | NIPS2019 | Code | Variational Denoising Network: Toward Blind Noise Modeling and Removal |
Image Denoising
https://github.com/wenbihan/reproducible-image-denoising-state-of-the-art/
reproducible-image-denoising-state-of-the-art
Collection of popular and reproducible image denoising works.
Criteria: works must have codes available, and the reproducible results demonstrate state-of-the-art performances.
This collection is inspired by the summary by flyywh
Note: This repo focuses on single image denoising in general, and will exclude multi-frame and video denoising works.
Denoising Algorithms
Filter
- NLM [Web] [Code] [PDF]
- A non-local algorithm for image denoising (CVPR 05), Buades et al.
- Image denoising based on non-local means filter and its method noise thresholding (SIVP2013), B. Kumar
- BM3D [Web] [Code] [PDF]
- Image restoration by sparse 3D transform-domain collaborative filtering (SPIE Electronic Imaging 2008), Dabov et al.
- PID [Web] [Code] [PDF]
- Progressive Image Denoising (TIP 2014), C. Knaus et al.
Sparse Coding
- KSVD [Web] [Code] [PDF]
- Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries (TIP 2006), Elad et al.
- LSSC [Web] [Code] [PDF]
- Non-local Sparse Models for Image Restoration (ICCV 2009), Mairal et al.
- NCSR [Web] [Code] [PDF]
- Nonlocally Centralized Sparse Representation for Image Restoration (TIP 2012), Dong et al.
- OCTOBOS [Web] [Code] [PDF]
- Structured Overcomplete Sparsifying Transform Learning with Convergence Guarantees and Applications (IJCV 2015), Wen et al.
- GSR [Web] [Code] [PDF]
- Group-based Sparse Representation for Image Restoration (TIP 2014), Zhang et al.
- TWSC [Web] [Code] [PDF]
- A Trilateral Weighted Sparse Coding Scheme for Real-World Image Denoising (ECCV 2018), Xu et al.
Classical External Priors
- EPLL [Web] [Code] [PDF]
- From Learning Models of Natural Image Patches to Whole Image Restoration (ICCV2011), Zoran et al.
- GHP [[Web]][Code] [PDF]
- Texture Enhanced Image Denoising via Gradient Histogram Preservation (CVPR2013), Zuo et al.
- PGPD [[Web]][Code] [PDF]
- Patch Group Based Nonlocal Self-Similarity Prior Learning for Image Denoising (ICCV 2015), Xu et al.
- PCLR [[Web]][Code] [PDF]
- External Patch Prior Guided Internal Clustering for Image Denoising (ICCV 2015), Chen et al.
Low Rank
- SAIST [Web] [Code by request] [PDF]
- Nonlocal image restoration with bilateral variance estimation: a low-rank approach (TIP2013), Dong et al.
- WNNM [Web] [Code] [PDF]
- Weighted Nuclear Norm Minimization with Application to Image Denoising (CVPR2014), Gu et al.
- Multi-channel WNNM [Web] [Code] [PDF]
- Multi-channel Weighted Nuclear Norm Minimization for Real Color Image Denoising (ICCV 2017), Xu et al.
Deep Denoising
- SF [Web] [Code] [PDF]
- Shrinkage Fields for Effective Image Restoration (CVPR 2014), Schmidt et al.
- TNRD [Web] [Code] [PDF]
- Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration (TPAMI 2016), Chen et al.
- RED [Web] [Code] [PDF]
- Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections (NIPS2016), Mao et al.
- DnCNN [Web] [Code] [PDF]
- Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising (TIP2017), Zhang et al.
- MemNet [Web] [Code] [PDF]
- MemNet: A Persistent Memory Network for Image Restoration (ICCV2017), Tai et al.
- WIN [Web] [Code] [PDF]
- Learning Pixel-Distribution Prior with Wider Convolution for Image Denoising (Arxiv), Liu et al.
- F-W Net [Web] [Code] [PDF]
- L_p-Norm Constrained Coding With Frank-Wolfe Network (Arxiv), Sun et al.
- NLCNN [Web] [Code] [PDF]
- Non-Local Color Image Denoising with Convolutional Neural Networks (CVPR 2017), Lefkimmiatis.
- xUnit [Web] [Code] [PDF]
- xUnit: Learning a Spatial Activation Function for Efficient Image Restoration (Arxiv), Kligvasser et al.
- UDNet [Web] [Code] [PDF]
- Universal Denoising Networks : A Novel CNN Architecture for Image Denoising (CVPR 2018), Stamatios Lefkimmiatis.
- Wavelet-CNN [Web] [Code] [PDF]
- Multi-level Wavelet-CNN for Image Restoration (Arxiv), Liu et al.
- FFDNet [Web] [Code] [PDF]
- FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising (TIP), Zhang et al.
- FC-AIDE [Web] [Code] [PDF]
- Fully Convolutional Pixel Adaptive Image Denoiser (Arxiv), Cha et al.
- CBDNet [Web] [Code] [PDF]
- Toward Convolutional Blind Denoising of Real Photographs (Arxiv), Guo et al.
- UDN [Web] [Code] [PDF]
- Universal Denoising Networks- A Novel CNN Architecture for Image Denoising (CVPR 2018), Lefkimmiatis.
- N3 [Web] [Code] [PDF]
- Neural Nearest Neighbors Networks (NIPS 2018), Plotz et al.
- NLRN [Web] [Code] [PDF]
- Non-Local Recurrent Network for Image Restoration (NIPS 2018), Liu et al.
- RDN+ [Web] [Code] [PDF]
- Residual Dense Network for Image Restoration (CVPR 2018), Zhang et al.
- FOCNet [Web] [Code] [PDF]
- FOCNet: A Fractional Optimal Control Network for Image Denoising (CVPR 2019), Jia et al.
Unsupervised / Weakly-Supervised Deep Denoising
- Noise2Noise [Web] [TF Code] [Keras Unofficial Code] [PDF]
- Noise2Noise: Learning Image Restoration without Clean Data (ICML 2018), Lehtinen et al.
- DIP [Web] [Code] [PDF]
- Deep Image Prior (CVPR 2018), Ulyanov et al.
- Noise2Void [Web] [Code] [PDF]
- Learning Denoising from Single Noisy Images (CVPR 2019), Krull et al.
- Noise2Self [Web] [Code] [PDF]
- LNoise2Self: Blind Denoising by Self-Supervision (ICML 2019), Batson and Royer
- Self-Supervised Denoising [Web] [Code] [PDF]
- High-Quality Self-Supervised Deep Image Denoising (NIPS 2019), Laine et al.
Real Noise Removal
- RIDNet [Web] [Code] [PDF]
- Real Image Denoising with Feature Attention (ICCV 2019), Anwar and Barnes.
- CBDNet [Web] [Code] [PDF]
- Real Image Denoising with Feature Attention (CVPR 2019), Guo et al.
- VDNNet [Web] [Code] [PDF]
- Variational Denoising Network: Toward Blind Noise Modeling and Removal (NIPS 2019), Yue et al.
Hybrid Model for Denoising
- STROLLR [PDF] [Code]
- When Sparsity Meets Low-Rankness: Transform Learning With Non-Local Low-Rank Constraint for Image Restoration (ICASSP 2017), Wen et al.
- Meets High-level Tasks [PDF] [Code]
- When Image Denoising Meets High-Level Vision Tasks: A Deep Learning Approach (IJCAI 2018), Liu et al.
- USA [PDF] [Code]
- Segmentation-aware Image Denoising Without Knowing True Segmentation (Arxiv), Wang et al.
Image Noise Level Estimation
- SINLE [PDF] [Code] [Slides]
- Single-image Noise Level Estimation for Blind Denoising (TIP 2014), Liu et al.
Novel Benchmark
- ReNOIR [Web] [Data] [PDF]
- RENOIR - A Dataset for Real Low-Light Image Noise Reduction (Arxiv 2014), Anaya, Barbu.
- Darmstadt [Web] [Data] [PDF]
- Benchmarking Denoising Algorithms with Real Photographs (CVPR 2017), Tobias Plotz, Stefan Roth.
- PolyU [Web] [Data] [PDF]
- Real-world Noisy Image Denoising: A New Benchmark (Arxiv), Xu et al.
Commonly Used Denoising Dataset
Commonly Used Image Quality Metrics
- PSNR (Peak Signal-to-Noise Ratio) [Wiki] [Matlab Code] [Python Code]
- SSIM (Structural similarity) [Wiki] [Matlab Code] [Python Code]
- NIQE (Naturalness Image Quality Evaluator) [Web] [Matlab Code] [Python Code]
