Please wait a minute...
Shopping Cart Email List Chinese
Advanced Search Fig/Tab
Frontiers of Computer Science

ISSN 2095-2228

ISSN 2095-2236(Online)

CN 10-1014/TP

Postal Subscription Code 80-970

2018 Impact Factor: 1.129

Featured Articles  |  Most Downloaded  |  Most Reading
Online Submission Subscribe to Our RSS Content Feed
Front. Comput. Sci.    2022, Vol. 16 Issue (5) : 165323    https://doi.org/10.1007/s11704-021-1010-8
RESEARCH ARTICLE
Self-corrected unsupervised domain adaptation
Yunyun WANG1(), Chao WANG1, Hui XUE2, Songcan CHEN3
1. School of Computer Science and Engineering, Nanjing University of Posts & Telecommunications, Nanjing 210046, China
2. School of Computer Science and Engineering, Southeast University, Nanjing 210096, China
3. School of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 210023, China
 Download: PDF(2345 KB)   HTML
 Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

Unsupervised domain adaptation (UDA), which aims to use knowledge from a label-rich source domain to help learn unlabeled target domain, has recently attracted much attention. UDA methods mainly concentrate on source classification and distribution alignment between domains to expect the correct target prediction. While in this paper, we attempt to learn the target prediction end to end directly, and develop a Self-corrected unsupervised domain adaptation (SCUDA) method with probabilistic label correction. SCUDA adopts a probabilistic label corrector to learn and correct the target labels directly. Specifically, besides model parameters, those target pseudo-labels are also updated in learning and corrected by the anchor-variable, which preserves the class candidates for samples. Experiments on real datasets show the competitiveness of SCUDA.
Keywords unsupervised domain adaptation      adversarial Learning      deep neural network      pseudo-labels      label corrector     
Corresponding Author(s): Yunyun WANG   
Just Accepted Date: 14 April 2021   Issue Date: 07 December 2021
 Cite this article:   
Yunyun WANG,Chao WANG,Hui XUE, et al. Self-corrected unsupervised domain adaptation[J]. Front. Comput. Sci., 2022, 16(5): 165323.
 URL:  
https://academic.hep.com.cn/fcs/EN/10.1007/s11704-021-1010-8
https://academic.hep.com.cn/fcs/EN/Y2022/V16/I5/165323
Fig.1  The architecture of SCUDA, which includes a feature extractor G, a domain discriminator D, a class classifier C, as well as a label corrector B with anchor-variable y~
Fig.2  
Methods A to W D to W W to D A to D D to A W to A Avg.
ResNet [36] 68.4±0.2 96.7±0.1 99.3±0.1 68.9±0.2 62.5±0.3 60.7±0.3 76.1
GFK [38] 72.8±0.0 95.0±0.0 98.2±0.0 74.5±0.0 63.4±0.0 61.0±0.0 77.5
TCA [37] 72.7±0.0 96.7±0.0 99.6±0.0 74.1±0.0 61.7±0.0 60.9±0.0 77.6
DAN [39] 80.5±0.4 97.1±0.2 99.6±0.1 78.6±0.2 63.6±0.3 62.8±0.2 80.4
RTN [18] 84.5±0.2 96.8±0.1 99.4±0.1 77.5±0.3 66.2±0.2 64.8±0.3 81.6
RevGrad [19] 82.0±0.4 96.9±0.2 99.1±0.1 79.7±0.4 68.2±0.4 67.4±0.5 82.2
ADDA [20] 86.2±0.5 96.2±0.3 98.4±0.3 77.8±0.3 69.5±0.4 68.9±0.5 82.9
JAN-A [40] 80.6±0.4 96.7±0.3 99.7±0.1 85.1±0.4 69.2±0.3 70.7±0.5 84.6
MADA [22] 90.0±0.1 97.4±0.1 99.6±0.1 87.8±0.2 70.3±0.3 66.4±0.3 85.2
iCAN [41] 92.5 98.8 100.0 90.1 72.1 69.9 87.2
SCUDA w/o Le 87.8±0.5 99.0±0.1 100.0±.0 88.9±0.6 73.3±0.5 70.3±0.4 86.6
SCUDA w/o Lc 82.0±0.3 98.7±0.0 100.0±.0 84.1±0.4 68.3±0.1 66.6±0.3 83.3
SCUDA 88.4±0.3 98.9±0.2 100.0±.0 90.2±0.4 74.4±0.4 71.7±0.4 87.3
Tab.1  Classification performance on the Office-31 dataset (%)
Methods I to P P to I I to C C to I C to P P to C Avg.
ResNet [36] 74.8±0.3 83.9±0.1 91.5±0.3 78.0±0.2 65.5±0.3 91.2±0.3 80.7
DAN [39] 74.5±0.4 82.2±0.2 92.8±0.2 86.3±0.4 69.2±0.4 89.8±0.4 82.5
RevGrad [19] 75.0±0.6 86.0±0.3 96.2±0.4 87.0±0.5 74.3±0.5 91.5±0.6 85.0
MADA [22] 75.0±0.3 87.9±0.2 96.0±0.3 88.8±0.3 75.2±0.2 92.2±0.3 85.8
iCAN [41] 79.5 89.7 94.7 89.9 78.5 92.0 87.4
SCUDA 79.7±0.3 88.4±0.4 96.8±0.2 89.3±0.2 79.1±0.2 92.9±0.4 87.7
Tab.2  Classification performance on the ImageCLEF-DA dataset (%)
Methods S to M U to M M to U
Source only 60.1±0.0 57.1±0.0 75.2±0.0
RevGrad [19] 73.9 73.0±0.0 77.1±0.0
ADDA [20] 76.0±1.8 89.4±0.2 90.1±0.8
MCD [26] 96.2±0.4 96.5±0.3 94.1±0.3
DeepJDOT [42] 96.7 95.7 96.4
CyCADA [43] 90.4±0.4 95.6±0.2 96.5±0.1
SCUDA 98.0±0.4 98.1±0.1 97.1±0.0
Tab.3  Classification performance on the digit dataset (%)
Fig.3  The t-SNE visualization of deep features extracted by RevGrad with (a) source=A, (b) target=W, (c) the merge of (a) and (b), and SCUDA with (d) source=A, (e) target=W, (f) the merge of (d) and (e)
Fig.4  Classification performance with the change of value k in task D to A
Fig.5  Error curve of ResNet, RevGrad and SCUDA
1 X C Li , D C Zhan , J Q Yang , Y Shi . Deep multiple instance selection. Science China Information Sciences, 2021, 64( 3): 130102–
2 S Y Li , S J Huang , S C Chen . Crowdsourcing aggregation with deep Bayesian learning. Science China Information Sciences, 2021, 64( 3): 130104–
3 M Xu , L Z Guo . Learning from group supervision: how supervision deficiency impacts multi-label learning. Science China Information Sciences, 2021, 64( 3): 130101–
4 X G Wang , J P Feng , W Y Liu . Deep graph cut network for weakly-supervised semantic segmentation. Science China Information Sciences, 2021, 64( 3): 130105–
5 X Zhao , N Pang , W Wang , W D Xiao , D K Guo . Few-shot text classification by leveraging bi-directional attention and cross-class knowledge. Science China Information Sciences, 2021, 64( 3): 130103–
6 J Ben-David , K S , A Blitzer , F Crammer , J W Kulesza . A theory of learning from different domains. Machine Learning, 2010, 79( 1−2): 151– 175
7 B C Sun, K Saenko. Deep coral: correlation alignment for deep domain adaptation. In: Proceedings of European Conference on Computer Vision. 2016, 443– 450
8 W Zellinger, T Grubinger, E Lughofer, T Natschläger, S Saminger-Platz. Central moment discrepancy (CMD) for domain-invariant representation learning. In: Proceedings of International Conference on Learning Representations. 2017
9 Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial nets. In: Proceedings of the 27th International Conference on Neural Information Processing Systems. 2014, 2672−2680
10 S J Pan , Q Yang . A survey on transfer learning. IEEE Transactions on Knowledge Data Engineering, 2009, 22( 10): 1345– 1359
11 A Iscen, G Tolias, Y Avrithis, O Chum. Label propagation for deep semi-supervised learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019, 5070−5079
12 E Tzeng, J Hoffman, N Zhang, K Saenko, T Darrell. Deep domain confusion: maximizing for domain invariance. 2014, arXiv preprint arXiv:1412.3474
13 M Ghifary, W B Kleijn, M J Zhang. Domain adaptive neural networks for object recognition. In: Proceedings of Pacific Rim International Conference on Artificial Intelligence. 2014, 898– 904
14 H Yan, Y K Ding, P H Li, Q L Wang, Y Xu, W M Zuo. Mind the class weight bias: weighted maximum mean discrepancy for unsupervised domain adaptation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2017, 2272−2281
15 Y Zhu , F Zhuang , J Wang , G Ke , Q He . Deep subdomain adaptation network for image classification. IEEE Transactions on Neural Networks and Learning Systems, 2020, 99 : 1– 10
16 K Saito, Y Ushiku, T Harada. Asymmetric tri-training for unsupervised domain adaptation. In: Proceedings of International Conference on Machine Learning. 2017, 2988−2997
17 X Zhang, F X Yu, S Chang, S J Wang. Deep transfer network: unsupervised domain adaptation. 2015, arXiv preprint arXiv: 1503.0059
18 M S Long, H Zhu, J M Wang, M I Jordan. Unsupervised domain adaptation with residual transfer networks. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. 2016, 136– 144
19 Y Ganin , E Ustinova , H Ajakan , P Germain , H Larochelle , F Laviolette , M Marchand , V Lempitsky . Domain-adversarial training of neural networks. The Journal of Machine Learning Research, 2016, 17( 1): 2096– 2030
20 E Tzeng, J Hoffman, K Saenko, T Darrell. Adversarial discriminative domain adaptation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2017, 7167−7176
21 S A Xie, Z B Zheng, L Chen, C Chen. Learning semantic representations for unsupervised domain adaptation. In: Proceedings of International Conference on Machine Learning. 2018, 5423−5432
22 Z Y Pei, Z J Cao, M S Long, J M Wang. Multi-adversarial domain adaptation. In: Proceedings of AAAI Conference on Artificial Intelligence. 2018
23 Y Y Wang , J M Gu , C Wang , S C Chen . Discrimination-aware domain adversarial neural network. Journal of Computer Science and Technology, 2020, 35( 2): 1– 9
24 S N Wang, X Y Chen, Y B Wang, M S Long, J M Wang. Progressive adversarial networks for fine-grained domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2020, 9213−9222
25 A Kumar, P Sattigeri, K Wadhawan, L Karlinsky, R Feris, B Freeman, G Wornell. Co-regularized alignment for unsupervised domain adaptation. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018, 9345−9356
26 K Saito, K Watanabe, Y Ushiku, T Harada. Maximum classifier discrepancy for unsupervised domain adaptation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2018, 3723−3732
27 C H Yu, J D Wang, Y Q Chen, M Y Huang. Transfer learning with dynamic adversarial adaptation network. In: Proceedings of International Conference on Data Mining. 2019, 778– 786
28 Y F Li , L Z Guo , Z H Zhou . Towards safe weakly supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43( 1): 334– 346
29 Y F Li , D M Liang . Safe semi-supervised learning: a brief introduction. Frontiers of Computer Science, 2019, 13( 4): 669– 676
30 K Yi, J X Wu. Probabilistic end-to-end noise correction for learning with noisy labels. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2019, 7017−7025
31 G H Wang, J Wu. Repetitive reprediction deep decipher for semi-supervised learning. In: Proceedings of the AAAI Conference on Artificial Intelligence. 2020, 6170– 6177
32 K Saenko, B Kulis, M Fritz, T Darrell. Adapting visual category models to new domains. In: Proceedings of European Conference on Computer Vision. 2010, 213– 226
33 Y Netzer, T Wang, A Coates, A Bissacco, B Wu, A Y Ng. Reading digits in natural images with unsupervised feature learning. In: Proceedings of NIPS Workshop on Deep Learning and Unsupervised Feature Learning. 2011
34 LeCun Y, Matan O, Boser B, Henderson D, Howard R E, Hubbard W, Jacket LD, Baird H S. Handwritten zip code recognition with multilayer networks. In: Proceedings of the 10th International Conference on Pattern Recognition. 1990, 35−40
35 Y LeCun , L Bottou , Y Bengio , P Haffner . Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998, 86( 11): 2278– 2324
36 K M He, X Y Zhang, S Q Ren, J Sun. Deep residual learning for image recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2016, 770– 778
37 S J Pan , L W Tsang , J T Kwok , Q Yang . Domain adaptation via transfer component analysis. IEEE Transactions on Neural Networks, 2010, 22( 2): 199– 210
38 B Q Gong, Y Shi, F Sha, K Grauman. Geodesic flow kernel for unsupervised domain adaptation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2012, 2066−2073
39 M S Long, Y Cao, J M Wang, M I Jordan. Learning transferable features with deep adaptation networks. In: Proceedings of International Conference on Machine Learning. 2015, 97−105
40 M S Long, H Zhu, J M Wnag, M I Jordan. Deep transfer learning with joint adaptation networks. In: Proceedings of International Conference on Machine Learning. 2017, 2208−2217
41 W C Zhang. Oouyang W L, Li W, Wu D. Collaborative and adversarial network for unsupervised domain adaptation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2018, 3801−3809
42 B Bhushan Damodaran, B Kellenberger, R Flamary, D Tuia, N Courty. Deepjdot: deep joint distribution optimal transport for unsupervised domain adaptation. In: Proceedings of European Conference on Computer Vision. 2018, 447– 463
43 J Hoffman, E Tzeng, T Park, J Y Zhu, K Isola, A A P, T Saenko. Cycada: cycle-consistent adversarial domain adaptation. In: Proceedings of International Conference on Machine Learning. 2018, 1989−1998
44 J Donahue, Y Q Jai, O Vinyals, J Hoffman, N Zhang, E Tzeng, T Darrell. Decaf: a deep convolutional activation feature for generic visual recognition. In: Proceedings of International Conference on Machine Learning. 2014, 647– 655
[1] Hongjia RUAN, Huihui SONG, Bo LIU, Yong CHENG, Qingshan LIU. Intellectual property protection for deep semantic segmentation models[J]. Front. Comput. Sci., 2023, 17(1): 171306-.
[2] Tian CHENG, Kunsong ZHAO, Song SUN, Muhammad MATEEN, Junhao WEN. Effort-aware cross-project just-in-time defect prediction framework for mobile apps[J]. Front. Comput. Sci., 2022, 16(6): 166207-.
[3] Jian-Hao LUO,Wang ZHOU,Jianxin WU. Image categorization with resource constraints: introduction, challenges and advances[J]. Front. Comput. Sci., 2017, 11(1): 13-26.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed