Adaptive network combination for single-image reflection removal: a domain generalization perspective

doi:10.1007/s11704-024-3582-6

Front. Comput. Sci.

2025, Vol. 19

Issue (1) : 191703 https://doi.org/10.1007/s11704-024-3582-6

Image and Graphics

Adaptive network combination for single-image reflection removal: a domain generalization perspective

Ming LIU^1,², Jianan PAN¹, Zifei YAN¹, Wangmeng ZUO¹(

), Lei ZHANG²

¹. School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China
². Department of Computing, The Hong Kong Polytechnic University, Hong Kong 999077, China

Download: PDF(1253 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Corresponding Author(s): Wangmeng ZUO

Just Accepted Date: 05 July 2024 Issue Date: 30 August 2024

Cite this article:

Ming LIU,Jianan PAN,Zifei YAN, et al. Adaptive network combination for single-image reflection removal: a domain generalization perspective[J]. Front. Comput. Sci., 2025, 19(1): 191703.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-024-3582-6
https://academic.hep.com.cn/fcs/EN/Y2025/V19/I1/191703

Fig.1 Illustration of the proposed two adaptive network combination (AdaNEC) methods, i.e., output fusion (

OF

) and network interpolation (

NI

)

Dataset		Acquisition method	Dataset partition	Camera model	Apertures range	Camera/Glass angle	Acquisition environment	Piece of Glass (Thickness)	# of Samples $T$ (and $I$ )	Alignment of $I$ and $T$
Syn_CEIL		synthetic	train	?	?	front	indoor + outdoor	?	7,643 ( $∞$ )	perfect
Syn_zhang		synthetic	train	?	?	front + oblique	indoor + outdoor	?	13,700 ( $∞$ )	perfect
Unaligned		real-world	train	a DSLR	?	front + oblique	indoor + outdoor	1 (?)	250	~20 pixels shift
Real89/20		real-world	train/test	Canon EOS 600D	f/1.6 ? f/2	front + oblique	indoor + outdoor	1 (?)	89/20	aligned
Nature200/20		real-world	train/test	Canon EOS 750D	f/4 ? f/16	front + oblique	indoor + outdoor	2 (3/8 mm)	200/20	aligned
SIR²	Wild	real-world	test	Nikon D5300	unconstrained	unconstrained	indoor + outdoor	3 (3/5/10 mm)	55	aligned
	Postcard				f/11 ? f/32	controlled	indoor		199
	Solid				f/11 ? f/32	controlled	indoor		200

Tab.1 Comparison between the collection configuration of commonly used reflection removal datasets

Fig.2 Structure of reflection type-aware weighting (RTAW) module.

k i

and

q

are vectors extracted by

F i

and

F

v i

means expertise level of the

i

th expert

G i

on image

I

, which can be used to calculate the weights

Method	Real20 (20)		Wild (55)		Postcard (199)		Solid (200)		Nature20 (20)		Average (474/494)
Method	PSNR↑	SSIM↑	PSNR↑	SSIM↑	PSNR↑	SSIM↑	PSNR↑	SSIM↑	PSNR↑	SSIM↑	PSNR↑	SSIM↑
ERRNet [5]	22.07	0.781	25.13	0.889	22.76	0.864	24.62	0.898	20.86	0.757	23.79	0.877
$E R R N e t O F$ w/o $w ˉ i$	22.70	0.791	24.60	0.865	23.15	0.871	24.91	0.895	20.46	0.756	24.04	0.877
$E R R N e t O F$	22.80	0.790	25.26	0.890	23.08	0.874	25.26	0.904	20.99	0.768	24.24	0.885
$E R R N e t N I$ w/o $w ˉ i$	22.37	0.787	24.63	0.865	23.14	0.874	24.75	0.894	20.53	0.757	23.96	0.878
$E R R N e t N I$	22.81	0.791	25.69	0.895	23.56	0.884	25.13	0.902	21.20	0.771	24.44	0.889
IBCLN [4]	21.86	0.762	24.71	0.886	23.39	0.875	24.87	0.893	23.57	0.783	24.08	0.875
$I B C L N O F$ w/o $w ˉ i$	22.53	0.794	25.44	0.888	24.19	0.888	25.05	0.899	24.59	0.818	24.63	0.886
$I B C L N O F$	22.52	0.789	25.77	0.897	24.27	0.889	25.24	0.900	24.74	0.820	24.78	0.888
$I B L C N N I$ w/o $w ˉ i$	22.41	0.793	25.20	0.887	23.52	0.887	24.77	0.897	24.56	0.816	24.21	0.885
$I B C L N N I$	22.04	0.782	25.35	0.894	23.34	0.887	24.85	0.897	24.59	0.818	24.17	0.885
RAGNet [7]	22.95	0.793	25.52	0.880	23.67	0.879	26.15	0.903	21.21	0.765	24.90	0.886
$R A G N e t O F$ w/o $w ˉ i$	23.32	0.806	25.87	0.895	25.08	0.900	26.19	0.907	21.40	0.775	25.57	0.898
$R A G N e t O F$	23.34	0.807	25.85	0.896	25.20	0.903	26.17	0.908	21.48	0.776	25.60	0.900
$R A G N e t N I$ w/o $w ˉ i$	23.43	0.810	25.52	0.887	23.91	0.884	26.26	0.906	21.31	0.770	25.07	0.891
$R A G N e t N I$	23.18	0.802	26.25	0.899	24.90	0.906	25.66	0.903	21.44	0.777	25.31	0.900
Dong et al. [10]	23.34	0.812	25.73	0.902	23.72	0.903	24.36	0.898	23.45	0.808	24.21	0.897
YTMT [11]	23.26	0.806	25.48	0.890	22.91	0.884	24.87	0.896	20.94	0.778	24.05	0.886
DSRNet [12]	24.23	0.820	25.68	0.896	24.56	0.908	26.28	0.914	?	?	25.40	0.905

Tab.2 Quantitative comparison against the state-of-the-art backbones. Note that the models marked by gray (i.e., ERRNet and RAGNet based models, YTMT) are not trained on Nature dataset. The results of DSRNet on Nature20 is not given since the model trained without Nature20 is not released

Fig.3 Visual comparison of ERRNet based AdaNEC methods

1	R E Parkin . A note on the extinction coefficient and absorptivity of glass. Solar Energy, 2015, 114: 196–197
2	Q, Zheng B, Shi J, Chen X, Jiang L Y, Duan A C Kot . Single image reflection removal with absorption effect. In: Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021, 13395−13404
3	X, Zhang R, Ng Q Chen . Single image reflection separation with perceptual losses. In: Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 4786−4794
4	C, Li Y, Yang K, He S, Lin J E Hopcroft . Single image reflection removal through cascaded refinement. In: Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, 3562−3571
5	K, Wei J, Yang Y, Fu D, Wipf H Huang . Single image reflection removal exploiting misaligned training data and network enhancements. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 8170−8179
6	R, Wan B, Shi L Y, Duan A H, Tan A C Kot . Benchmarking single-image reflection removal algorithms. In: Proceedings of 2017 IEEE International Conference on Computer Vision. 2017, 3942−3950
7	Y, Li M, Liu Y, Yi Q, Li D, Ren W Zuo . Two-stage single image reflection removal with reflection-aware guidance. Applied Intelligence, 2023, 53( 16): 19433–19448
8	I J Good . Rational decisions. Journal of the Royal Statistical Society: Series B (Methodological), 1952, 14( 1): 107–114
9	S, Mannor D, Peleg R Rubinstein . The cross entropy method for classification. In: Proceedings of the 22nd International Conference on Machine Learning. 2005, 561−568
10	Z, Dong K, Xu Y, Yang H, Bao W, Xu R W H Lau . Location-aware single image reflection removal. In: Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. 2021, 4997−5006
11	Q, Hu X Guo . Trash or treasure? An interactive dual-stream strategy for single image reflection separation. In: Proceedings of the 35th International Conference on Neural Information Processing Systems. 2021, 1890
12	Q, Hu X Guo . Single image reflection separation via component synergy. In: Proceedings of 2023 IEEE/CVF International Conference on Computer Vision. 2023, 13092−13101

[1]	FCS-23582-OF-ML_suppl_1	Download
[2]	FCS-23582-OF-ML_suppl_2	Download

Viewed

Full text

Abstract

Cited

Shared

Discussed