Learning multi-tasks with inconsistent labels by using auxiliary big task
Quan FENG1,2, Songcan CHEN1,2()
1. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China 2. MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China
Multi-task learning is to improve the performance of the model by transferring and exploiting common knowledge among tasks. Existing MTL works mainly focus on the scenario where label sets among multiple tasks (MTs) are usually the same, thus they can be utilized for learning across the tasks. However, the real world has more general scenarios in which each task has only a small number of training samples and their label sets are just partially overlapped or even not. Learning such MTs is more challenging because of less correlation information available among these tasks. For this, we propose a framework to learn these tasks by jointly leveraging both abundant information from a learnt auxiliary big task with sufficiently many classes to cover those of all these tasks and the information shared among those partially-overlapped tasks. In our implementation of using the same neural network architecture of the learnt auxiliary task to learn individual tasks, the key idea is to utilize available label information to adaptively prune the hidden layer neurons of the auxiliary network to construct corresponding network for each task, while accompanying a joint learning across individual tasks. Extensive experimental results demonstrate that our proposed method is significantly competitive compared to state-of-the-art methods.
Qingyong Zheng and Ya Gao contributed equally to this work.
引用本文:
. [J]. Frontiers of Computer Science, 2023, 17(5): 175342.
Quan FENG, Songcan CHEN. Learning multi-tasks with inconsistent labels by using auxiliary big task. Front. Comput. Sci., 2023, 17(5): 175342.
A, Khattar S, Hegde R Hebbalaguppe . Cross-domain multi-task learning for object detection and saliency estimation. In: Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2021, 3634−3643
2
L, Al-Qaisi M A, Hassonah M M, Al-Zoubi A M Al-Zoubi . A review of evolutionary data clustering algorithms for image segmentation. In: Aljarah I, Faris H, Mirjalili S, eds. Evolutionary Data Clustering: Algorithms and Applications. Singapore: Springer, 2021, 201−214
3
I, Chaturvedi C L, Su R E Welsch . Fuzzy aggregated topology evolution for cognitive multi-tasks. Cognitive Computation, 2021, 13( 1): 96–107
4
M, Ravanelli J, Zhong S, Pascual P, Swietojanski J, Monteiro J, Trmal Y Bengio . Multi-task self-supervised learning for robust speech recognition. In: Proceedings of 2020 IEEE International Conference on Acoustics, Speech and Signal Processing. 2020, 6989−6993
5
C, Li B, Wang S, Zhang Y, Liu R, Song J, Cheng X Chen . Emotion recognition from EEG based on multi-task learning with capsule network and attention mechanism. Computers in Biology and Medicine, 2022, 143: 105303
6
B, Cheng B, Zhu S Pu . Multi-auxiliary domain transfer learning for diagnosis of MCI conversion. Neurological Sciences, 2022, 43( 3): 1721–1739
7
S, Ruder J, Bingel I, Augenstein A Søgaard . Sluice networks: learning what to share between loosely related tasks. 2017, arXiv preprint arXiv: 1705.08142v1
8
T, Gong X, Zheng X Lu . Remote sensing scene classification with multi-task learning. In: Wang L, Wu Y, Gong J, eds. Proceedings of the 7th China High Resolution Earth Observation Conference (CHREOC 2020). Singapore: Springer, 2022, 403−418
9
M M, Islam T Iqbal . Mumu: cooperative multitask learning-based guided multimodal fusion. In: Proceedings of the 36th Conference on Artificial Intelligence. 2022, 1043−1051
10
M, Long Z, Cao J, Wang P S Yu . Learning multiple tasks with multilinear relationship networks. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 1593−1602
11
J, Ma Z, Zhao X, Yi J, Chen L, Hong E H Chi . Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018, 1930−1939
12
Q, Feng J, Yao Y, Zhong P, Li Z Pan . Learning twofold heterogeneous multi-task by sharing similar convolution kernel pairs. Knowledge-Based Systems, 2022, 252: 109396
13
S, Wu H R, Zhang C Ré . Understanding and improving information transfer in multi-task learning. In: Proceedings of the 8th International Conference on Learning Representations. 2020, 26−30
14
T, Evgeniou M Pontil . Regularized multi-task learning. In: Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2004, 109−117
15
J, Honorio D Samaras . Multi-task learning of Gaussian graphical models. In: Proceedings of the 27th International Conference on Machine Learning. 2010, 447−454
16
Q, Liu X, Li Z, He N, Fan D, Yuan W, Liu Y Liang . Multi-task driven feature models for thermal infrared tracking. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020, 11604−11611
17
J, Wang S, Zhang Y, Wang Z Zhu . Learning efficient multi-task stereo matching network with richer feature information. Neurocomputing, 2021, 421: 151–160
18
P, Guo C, Deng L, Xu X, Huang Y Zhang . Deep multi-task augmented feature learning via hierarchical graph neural network. In: Proceedings of European Conference on Machine Learning and Knowledge Discovery in Databases. 2021, 538−553
19
S, Vandenhende S, Georgoulis Gansbeke W, van M, Proesmans D, Dai Gool L van . Multi-task learning for dense prediction tasks: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44( 7): 3614–3633
20
Z, Shen C, Cui J, Huang J, Zong M, Chen Y Yin . Deep adaptive feature aggregation in multi-task convolutional neural networks. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020, 2213−2216
21
S, Yadav J, Chauhan J P, Sain K, Thirunarayan A, Sheth J Schumm . Identifying depressive symptoms from tweets: figurative language enabled multitask learning framework. In: Proceedings of the 28th International Conference on Computational Linguistics. 2020, 696−709
22
X, Sun R, Panda R, Feris K Saenko . Adashare: learning what to share for efficient deep multi-task learning. In: Proceedings of the 34th Conference on Neural Information Processing Systems. 2020, 8728−8740
23
T, Sun Y, Shao X, Li P, Liu H, Yan X, Qiu X Huang . Learning sparse sharing architectures for multiple tasks. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020, 8936−8943
24
S, Verboven Chaudhary M, Hafeez J, Berrevoets W Verbeke . HydaLearn: highly dynamic task weighting for multi-task learning with auxiliary tasks. 2008, arXiv preprint arXiv: 2008.11643
25
V, Sanh T, Wolf S Ruder . A hierarchical multi-task approach for learning embeddings from semantic tasks. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 6949−6956
26
J Baxter . A Bayesian/information theoretic model of learning to learn via multiple task sampling. Machine Learning, 1997, 28( 1): 7–39
27
S, Ruder J, Bingel I, Augenstein A Søgaard . Latent multi-task architecture learning. In: In Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 4822−4829
28
G, Strezoski N, Noord M Worring . Many task learning with task routing. In: Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. 2019, 1375−1384
29
C, Fernando D, Banarse C, Blundell Y, Zwols D, Ha A A, Rusu A, Pritzel D Wierstra . PathNet: Evolution channels gradient descent in super neural networks. 2017, arXiv preprint arXiv: 1701.08734
30
G, Pironkov S U, Wood S Dupont . Hybrid-task learning for robust automatic speech recognition. Computer Speech & Language, 2020, 64: 101103
31
P, Cao X, Shan D, Zhao M, Huang O Zaiane . Sparse shared structure based multi-task learning for MRI based cognitive performance prediction of Alzheimer’s disease. Pattern Recognition, 2017, 72: 219–235
32
S, Lee Y Son . Multitask learning with single gradient step update for task balancing. Neurocomputing, 2022, 467: 442–453
33
A, Søgaard Y Goldberg . Deep multi-task learning with low level tasks supervised at lower layers. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 231−235
34
J, Fan T, Zhao Z, Kuang Y, Zheng J, Zhang J, Yu J Peng . HD-MTL: hierarchical deep multi-task learning for large-scale visual recognition. IEEE Transactions on Image Processing, 2017, 26( 4): 1923–1938
35
F, Ott D, Rügamer L, Heublein B, Bischl C Mutschler . Joint classification and trajectory regression of online handwriting using a multi-task learning approach. In: Proceedings of 2022 IEEE/CVF Winter Conference on Applications of Computer Vision. 2022, 1244−1254
36
C, Zhang Y, Li N, Du W, Fan P S Yu . Joint slot filling and intent detection via capsule neural networks. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019, 5259−5267
37
X C, Li D C Zhan . FedRS: federated learning with restricted softmax for label distribution non-IID data. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021, 995−1005
38
Y, Wang Z, Zhang W, Hao C Song . Attention guided multiple source and target domain adaptation. IEEE Transactions on Image Processing, 2021, 30: 892–906
39
W Xue . Weighted feature-task-aware regularization learner for multitask learning. Pattern Analysis and Applications, 2020, 23( 1): 253–263
40
J, Zhang J, Miao K, Zhao Y Tian . Multi-task feature selection with sparse regularization to extract common and task-specific features. Neurocomputing, 2019, 340: 76–89
41
W, Shao Y, Peng C, Zu M, Wang D, Zhang Alzheimer's Disease Neuroimaging Initiative The . Hypergraph based multi-task feature selection for multimodal classification of Alzheimer’s disease. Computerized Medical Imaging and Graphics, 2020, 80: 101663
42
L, Li X, Pan H, Yang Z, Liu Y, He Z, Li Y, Fan Z, Cao L Zhang . Multi-task deep learning for fine-grained classification and grading in breast cancer histopathological images. Multimedia Tools and Applications, 2020, 79(21−22): 14509−14528
43
Z, Zheng Y, Wang Q, Dai H, Zheng D Wang . Metadata-driven task relation discovery for multi-task learning. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 4426−4432
44
C, Yan J, Xu J, Xie C, Cai H Lu . Prior-aware CNN with multi-task learning for colon images analysis. In: Proceedings of the 17th IEEE International Symposium on Biomedical Imaging. 2020, 254−257
45
I, Misra A, Shrivastava A, Gupta M Hebert . Cross-stitch networks for multi-task learning. In: Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016, 3994−4003
46
R, Duan N F Chen . Unsupervised feature adaptation using adversarial multi-task training for automatic evaluation of children’s speech. In: Proceedings of the 21st Annual Conference of the International Speech Communication Association. 2020, 3037−3041
47
I, Augenstein A Søgaard . Multi-task learning of keyphrase boundary classification. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017, 341−346
48
P, Rai III H Daumé . Infinite predictor subspace models for multitask learning. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics. 2010, 613−620
49
L, Zhou Z, Cui C, Xu Z, Zhang C, Wang T, Zhang J Yang . Pattern-structure diffusion for multi-task learning. In: Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, 4513−4522
50
Y, Wang X, Luo L, Ding S, Fu S Hu . Multi-task non-negative matrix factorization for visual object tracking. Pattern Analysis and Applications, 2020, 23( 1): 493–507
51
J Y, Jeong C H Jun . Sparse tensor decomposition for multi-task interaction selection. In: Proceedings of 2019 IEEE International Conference on Big Knowledge. 2019, 105−114
52
F, Huang Y, Qiu Q, Li S, Liu F Ni . Predicting drug-disease associations via multi-task learning based on collective matrix factorization. Frontiers in Bioengineering and Biotechnology, 2020, 8: 218
53
Y, Zhang Y, Zhang W Wang . Deep multi-task learning via generalized tensor trace norm. 2002, arXiv preprint arXiv: 2002.04799
54
Z, Chen H, Lei Y, Zhao Z, Huang X, Xiao Y, Lei E L, Tan B Lei . Template-oriented multi-task sparse low-rank learning for parkinson’s diseases diagnosis. In: Proceedings of the 3rd International Workshop on PRedictive Intelligence in MEdicine. 2020, 178−187
55
X, Wu X, Zhang Y Cen . Multi-task joint sparse and low-rank representation target detection for hyperspectral image. IEEE Geoscience and Remote Sensing Letters, 2019, 16( 11): 1756–1760
56
Y, Zhang Q Yang . A survey on multi-task learning. IEEE Transactions on Knowledge and Data Engineering, 2021
57
K, He X, Chen S, Xie Y, Li P, Dollár R Girshick . Masked autoencoders are scalable vision learners. In: Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, 15979−15988
58
K, Simonyan A Zisserman . Very deep convolutional networks for large-scale image recognition. In: Proceedings of the 3rd International Conference on Learning Representations. 2015, 1556
59
J, Jaworek-Korjakowska P, Kleczek M Gorgon . Melanoma thickness prediction based on convolutional neural network with VGG-19 model transfer learning. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2019, 2748−2756
60
Y, Gao J, Ma M, Zhao W, Liu A L Yuille . Nddr-CNN: layerwise feature fusing in multi-task CNNs by neural discriminative dimensionality reduction. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 3200−3209
61
F, Heuer S, Mantowsky S S, Bukhari G Schneider . Multitask-centerNet (MCN): efficient and diverse multitask learning using an anchor free approach. In: Proceedings of 2021 IEEE/CVF International Conference on Computer Vision Workshops. 2021, 997−1005
