1. School of Computer Science, Nanjing University of Posts and Telecommunications, Nanjing 210023, China 2. Jiangsu Key Laboratory of Big Data Security & Intelligent Processing, Nanjing University of Posts and Telecommunications, Nanjing 210023, China 3. College of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing 210003, China
Image super-resolution (SR) is one of the classic computer vision tasks. This paper proposes a super-resolution network based on adaptive frequency component upsampling, named SR-AFU. The network is composed of multiple cascaded dilated convolution residual blocks (CDCRB) to extract multi-resolution features representing image semantics, and multiple multi-size convolutional upsampling blocks (MCUB) to adaptively upsample different frequency components using CDCRB features. The paper also defines a new loss function based on the discrete wavelet transform, making the reconstructed SR images closer to human perception. Experiments on the benchmark datasets show that SR-AFU has higher peak signal to noise ratio (PSNR), significantly faster training speed and more realistic visual effects compared with the existing methods.
W T Freeman , E C Pasztor , O T Carmichael . Learning low-level vision. International Journal of Computer Vision, 2000, 40( 1): 25– 47
2
W Shi, J Caballero, C Ledig, X Zhuang, W Bai, K Bhatia, Marvao A M S M de, T Dawes, D O’Regan, D Rueckert. Cardiac image super-resolution with global correspondence using multi-atlas patchmatch. In: Proceedings of the 16th International Conference on Medical Image Computing and Computer-Assisted Intervention. 2013, 9−16
3
S Zhang , G Liang , S Pan , L Zheng . A fast medical image super resolution method based on deep learning network. IEEE Access, 2018, 7 : 12319– 12327
4
J Oh , C Lee , D C Seo . Automated HRSI georegistration using orthoimage and SRTM: focusing KOMPSAT-2 imagery. Computers & Geosciences, 2013, 52 : 77– 84
5
K Nogueira , O A B Penatti , J A Dos Santos . Towards better exploiting convolutional neural networks for remote sensing scene classification. Pattern Recognition, 2017, 61 : 539– 556
6
X Hu , P Ma , Z Mai , S Peng , Z Yang , L Wang . Face hallucination from low quality images using definition-scalable inference. Pattern Recognition, 2019, 94 : 110– 121
7
L C Chen, Y Zhu, G Papandreou, F Schroff, H Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the 15th European Conference on Computer Vision. 2018, 833−851
8
P Li, Q Wang, W Zuo, L Zhang. Log-Euclidean kernels for sparse representation and dictionary learning. In: Proceedings of IEEE International Conference on Computer Vision. 2013, 1601−1608
9
Y Lee, Y Choe. Neighbor embedding based single image super-resolution using hybrid feature and adaptive weight decay regularization. In: Proceedings of the 4th IEEE International Conference on Consumer Electronics Berlin. 2014, 185−187
10
C Dong , C C Loy , K He , X Tang . Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38( 2): 295– 307
11
J Kim, J K Lee, K M Lee. Accurate image super-resolution using very deep convolutional networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2016, 1646−1654
12
J Kim, J K Lee, K M Lee. Deeply-recursive convolutional network for image super-resolution. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2016, 1637−1645
13
Y Zhang, Y Tian, Y Kong, B Zhong, Y Fu. Residual dense network for image super-resolution. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 2472−2481
14
Y Zhang, K Li, K Li, L Wang, B Zhong, Y Fu. Image super-resolution using very deep residual channel attention networks. In: Proceedings of the 15th European Conference on Computer Vision. 2018, 294−310
15
M S Rad, B Bozorgtabar, U V Marti, M Basler, H K Ekenel, J P Thiran. SROBB: targeted perceptual loss for single image super-resolution. In: Proceedings of IEEE/CVF International Conference on Computer Vision. 2019, 2710−2719
16
M K Ng , H Shen , E Y Lam , L Zhang . A total variation regularization based super-resolution reconstruction algorithm for digital video. EURASIP Journal on Advances in Signal Processing, 2007, 2007 : 074585–
17
K Jiang , Z Wang , P Yi , J Jiang . Hierarchical dense recursive network for image super-resolution. Pattern Recognition, 2020, 107 : 107475–
18
C Dong, C C Loy, X Tang. Accelerating the super-resolution convolutional neural network. In: Proceedings of the 14th European Conference on Computer Vision. 2016, 391−407
19
K He, X Zhang, S Ren, J Sun. Deep residual learning for image recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2016, 770−778
20
W S Lai, J B Huang, N Ahuja, M H Yang. Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2017, 5835−5843
21
C Ledig, L Theis, F Huszár, J Caballero, A Cunningham, A Acosta, A Aitken, A Tejani, J Totz, Z Wang, W Shi. Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2017, 105−114
22
X Wang, K Yu, S Wu, J Gu, Y Liu, C Dong, Y Qiao, C C Loy. ESRGAN: enhanced super-resolution generative adversarial networks. In: Proceedings of the European Conference on Computer Vision (ECCV) Workshops. 2018, 63−79
23
B Lim, S Son, H Kim, S Nah, K M Lee. Enhanced deep residual networks for single image super-resolution. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2017, 1132−1140
24
J Yu, Y Fan, J Yang, N Xu, Z Wang, X Wang, T Huang. Wide activation for efficient and accurate image super-resolution. 2018, arXiv preprint arXiv: 1808.08718
25
T Dai, J Cai, Y Zhang, S T Xia, L Zhang. Second-order attention network for single image super-resolution. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 11057−11066
26
H Zhao, O Gallo, I Frosio, J Kautz. Loss functions for neural networks for image processing. 2018, arXiv preprint arXiv: 1511.08861
27
R Timofte, E Agustsson, L Van Gool, M H Yang, L Zhang. NTIRE 2017 challenge on single image super-resolution: methods and results. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2017, 1110−1121
28
M Bevilacqua, A Roumy, C Guillemot, M L A Morel. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of British Machine Vision Conference. 2012
29
R Zeyde, M Elad, M Protter. On single image scale-up using sparse-representations. In: Proceedings of the 7th International Conference on Curves and Surfaces. 2010, 711−730
30
D Martin, C Fowlkes, D Tal, J Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the 8th IEEE International Conference on Computer Vision. 2001, 416−423
31
J B Huang, A Singh, N Ahuja. Single image super-resolution from transformed self-exemplars. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2015, 5197−5206
32
D P Kingma, J Ba. Adam: a method for stochastic optimization. In: Proceedings of the 3rd International Conference for Learning Representations. 2015
33
Z Wang , A C Bovik , H R Sheikh , E P Simoncelli . Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13( 4): 600– 612
34
C Dong, C C Loy, K He, X Tang. Learning a deep convolutional network for image super-resolution. In: Proceedings of the 13th European Conference on Computer Vision. 2014, 184−199
35
Y Tai, J Yang, X Liu, C Xu. MemNet: a persistent memory network for image restoration. In: Proceedings of IEEE International Conference on Computer Vision. 2017, 4549−4557
36
M Haris, G Shakhnarovich, N Ukita. M Haris, G Shakhnarovich, N Ukita. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 1664−1673
37
Z Li, J Yang, Z Liu, X Yang, G Jeon, W Wu. Feedback network for image super-resolution. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 3862−3871
38
Z Hui, X Gao, Y Yang, X Wang. Lightweight image super-resolution with information multi-distillation network. In: Proceedings of the 27th ACM International Conference on Multimedia. 2019, 2024−2032
39
H Zhao, X Kong, J He, Y Qiao, C Dong. Efficient image super-resolution using pixel attention. In: Proceedings of the European Conference on Computer Vision. 2020, 56−72
40
C Wang, Z Li, J Shi. Lightweight image super-resolution with adaptive weighted learning network. 2019, arXiv preprint arXiv: 1904.02358
