1. School of Information Science and Technology, Beijing Forestry University, Beijing 100083, China 2. Engineering Research Center for Forestry Oriented Intelligent Information Processing, National Forestry and Grassland Administration, Beijing 100083, China 3. School of Information Science and Technology, North China University of Technology, Beijing 100144, China 4. College of Computer Science and Technology, Inner Mongolia Normal University, Hohhot 010020, China 5. Department of Computer Science and Software Engineering, Swinburne University of Technology, Hawthorn VIC 3122, Australia 6. Key Laboratory of High Confidence Software Technologies (Peking University), Ministry of Education, Beijing 100871, China 7. Institute of Software, School of Computer Science, Peking University, Beijing 100871, China
Agile development aims at rapidly developing software while embracing the continuous evolution of user requirements along the whole development process. User stories are the primary means of requirements collection and elicitation in the agile development. A project can involve a large amount of user stories, which should be clustered into different groups based on their functionality’s similarity for systematic requirements analysis, effective mapping to developed features, and efficient maintenance. Nevertheless, the current user story clustering is mainly conducted in a manual manner, which is time-consuming and subjective to human bias. In this paper, we propose a novel approach for clustering the user stories automatically on the basis of natural language processing. Specifically, the sentence patterns of each component in a user story are first analysed and determined such that the critical structure in the representative tasks can be automatically extracted based on the user story meta-model. The similarity of user stories is calculated, which can be used to generate the connected graph as the basis of automatic user story clustering. We evaluate the approach based on thirteen datasets, compared against ten baseline techniques. Experimental results show that our clustering approach has higher accuracy, recall rate and F1-score than these baselines. It is demonstrated that the proposed approach can significantly improve the efficacy of user story clustering and thus enhance the overall performance of agile development. The study also highlights promising research directions for more accurate requirements elicitation.
. [J]. Frontiers of Computer Science, 2023, 17(6): 176213.
Bo YANG, Xiuyin MA, Chunhui WANG, Haoran GUO, Huai LIU, Zhi JIN. User story clustering in agile development: a framework and an empirical study. Front. Comput. Sci., 2023, 17(6): 176213.
As a job seeker, I want to search jobs with basic criteria, so that I can find a suitable job.
Browse Jobs
Find Job
1
As a job seeker, I want to select desired working location, so that I can save transportation costs.
2
As a job seeker, I want to upload resume in PDF, so that others can view my resume online.
Post Resume
3
As a job seeker, I want to upload resume in Word,so that others can give me some advice and help me to amend it.
Tab.1
Fig.1
Fig.2
Fig.3
Fig.4
Dataset
# Stories
# Nouns
# Verbs
Data set description
saweb1
87
13
41
A collection of user stories written to describe the features of earlier versions of the Scrum Alliance website
ctc2
72
17
38
Describes the needs of online video learning training sites
repBath3
53
4
23
Describes the first version of the user story at www.PlanningPoker.com, a site that allows geographically dispersed evaluators to work together
g02_federalspending
97
23
34
Describe the need to manage a company’s financial expenditure website
g03_loudoun
56
20
29
Describe the requirements for managing government affairs software
g04_recycling
51
18
20
Describe the need to manage classified recycling sites
g05_openspending
53
13
16
Describe the requirements of an open source data sharing platform
g08_frictionless
66
5
22
Describe the needs of data analysis tools
g10_scrumalliance
97
23
33
Describe scrum training website requirements (manually classified version of saweb1)
g11_nsf
73
29
22
Describe software requirements for management company development teams
g12_camperplus
55
23
24
Describe the requirements for managing summer camp members and event software
g13_planningpoker
53
20
19
Describe the requirements of PlanningPoker website (manually classified version of repBath3)
g14_datahub
67
12
30
Describe the requirements of a data hosting platform
Tab.2
Semantic similarity prototype
CUSNLP-Noun
CUSNLP-Verb
saweb1
0.1427
0.6128
0.5782
ctc2
0.1907
0.6918
0.6744
repBath3
0.3791
0.5569
0.6535
g02_federalspending
0.3076
0.7070
0.6846
g03_loudoun
0.2816
0.5172
0.6724
g04_recycling
0.1828
0.5434
0.6915
g05_openspending
0.2998
0.6164
0.5211
g08_frictionless
0.3333
0.4756
0.6329
g10_scrumalliance
0.1610
0.7555
0.6202
g11_nsf
0.1526
0.7149
0.5737
g12_camperplus
0.3569
0.6379
0.6992
g13_planningpoker
0.2808
0.3740
0.6604
g14_datahub
0.2449
0.4614
0.6120
Tab.3
Pair of methods
Effect size
CUSNLP-Noun vs. semantic similarity prototype
1.7053
CUSNLP-Verb vs. semantic similarity prototype
1.8529
CUSNLP-Verb vs. CUSNLP-Noun
0.5188
Tab.4
saweb1
ctc2
repBath3
g02
g03
g04
g05
g08
g10
g11
g12
g13
g14
Average
TF-IDF+K-means
0.5057
0.5921
0.5011
0.5806
0.5696
0.3915
0.2980
0.668
0.3513
0.5029
0.668
0.3513
0.5029
0.5367
TF-IDF+DBSCAN
0.4809
0.336
0.5418
0.336
0.2620
0.2899
0.4924
0.3522
0.4363
0.3426
0.2823
0.4068
0.3946
0.3811
TF-IDF+Spectral
0.4217
0.5661
0.5733
0.5418
0.5733
0.6362
0.5790
0.5132
0.5662
0.6283
0.6526
0.6658
0.5724
0.5761
Doc2Vec+K-means
0.3375
0.4638
0.6537
0.4085
0.3986
0.6265
0.6271
0.3279
0.1960
0.4562
0.3456
0.6548
0.3711
0.4513
Doc2Vec+DBSCAN
0.2018
0.2592
0.3870
0.1000
0.1775
0.2000
0.4108
0.4108
0.1947
0.2384
0.0831
0.0951
0.2955
0.2349
Doc2Vec+Spectral
0.1422
0.2045
0.3945
0.5243
0.3164
0.2752
0.6408
0.325
0.3132
0.3234
0.5585
0.5484
0.2791
0.3727
TCNN+K-means
0.6632
0.7775
0.5529
0.4582
0.5257
0.6641
0.5540
0.5524
0.2363
0.4556
0.4127
0.6976
0.5650
0.5473
TCNN+DBSCAN
0.5070
0.456
0.7565
0.2978
0.2941
0.4621
0.4603
0.4068
0.2402
0.1731
0.3106
0.3978
0.4508
0.4010
TCNN+Spectral
0.8404
0.8636
0.7553
0.4094
0.5937
0.7373
0.6800
0.4517
0.2428
0.5393
0.3882
0.6166
0.5244
0.5879
CUSNLP-Noun
0.6128
0.6918
0.5569
0.7070
0.5172
0.5434
0.6164
0.4756
0.7555
0.7149
0.6379
0.3740
0.4614
0.5896
CUSNLP-Verb
0.5782
0.6744
0.6535
0.6846
0.6724
0.6915
0.5211
0.6329
0.6202
0.5737
0.6992
0.6604
0.6120
0.6365
Tab.5
Pair of methods
Effect size
CUSNLP-Noun vs. TF-IDF+K-means
0.4264
CUSNLP-Noun vs. TF-IDF+DBSCAN
1.4339
CUSNLP-Noun vs. TF-IDF+Spectral
0.1482
CUSNLP-Noun vs. Doc2Vec+K-means
0.9413
CUSNLP-Noun vs. Doc2Vec+DBSCAN
1.6680
CUSNLP-Noun vs. Doc2Vec+Spectral
1.2666
CUSNLP-Noun vs. TCNN+K-means
0.3335
CUSNLP-Noun vs. TCNN+DBSCAN
1.1741
CUSNLP-Noun vs. TCNN+Spectral
0.0113
CUSNLP-Verb vs. TF-IDF+K-means
0.8949
CUSNLP-Verb vs. TF-IDF+DBSCAN
1.7233
CUSNLP-Verb vs. TF-IDF+Spectral
0.9200
CUSNLP-Verb vs. Doc2Vec+K-means
1.2875
CUSNLP-Verb vs. Doc2Vec+DBSCAN
1.8032
CUSNLP-Verb vs. Doc2Vec+Spectral
1.5143
CUSNLP-Verb vs. TCNN+K-means
0.7870
CUSNLP-Verb vs. TCNN+DBSCAN
1.4567
CUSNLP-Verb vs. TCNN+Spectral
0.3561
Tab.6
Fig.5
Fig.6
saweb1
ctc2
repBath3
g02_federalspending
TF-IDF+K-means
0.191
0.312
0.23
0.309
TF-IDF+DBSCAN
0.042
0.017
0.011
0026
TF-IDF+Spectral
0.296
0.1241
0.048
0.139
Doc2Vec+K-means
6.287
5.37
5.37
5.902
Doc2Vec+DBSCAN
6.293
5.482
4.819
5.899
Doc2Vec+Spectral
6.04
5.461
4.988
6.453
TCNN+K-means
1548.746
1548.65
1548.584
1548.873
TCNN+DBSCAN
1548.563
1548.511
1548.448
1548.696
TCNN+Spectral
1547.581
1547.471
1547.398
1547.583
CUSNLP-Noun
6.667
2.07
17.616
14.895
CUSNLP-Verb
2.042
0.113
0.468
4.577
g03_loudoun
g04_recycling
g05_openspending
TF-IDF+K-means
0.5696
0.3915
0.2980
TF-IDF+DBSCAN
0.2620
0.2899
0.4924
TF-IDF+Spectral
0.5733
0.6362
0.5790
Doc2Vec+K-means
0.3986
0.6265
0.6271
Doc2Vec+DBSCAN
0.1775
0.2000
0.4108
Doc2Vec+Spectral
0.3164
0.2752
0.6408
TCNN+K-means
0.5257
0.6641
0.5540
TCNN+DBSCAN
0.2941
0.4621
0.4603
TCNN+Spectral
0.5937
0.7373
0.6800
CUSNLP-Noun
0.5172
0.5434
0.6164
CUSNLP-Verb
0.6724
0.6915
0.5211
g08_frictionless
g10_scrumalliance
g11_nsf
TF-IDF+K-means
0.668
0.3513
0.5029
TF-IDF+DBSCAN
0.3522
0.4363
0.3426
TF-IDF+Spectral
0.5132
0.5662
0.6283
Doc2Vec+K-means
0.3279
0.1960
0.4562
Doc2Vec+DBSCAN
0.4108
0.1947
0.2384
Doc2Vec+Spectral
0.325
0.3132
0.3234
TCNN+K-means
0.5524
0.2363
0.4556
TCNN+DBSCAN
0.4068
0.2402
0.1731
TCNN+Spectral
0.4517
0.2428
0.5393
CUSNLP-Noun
0.4756
0.7555
0.7149
CUSNLP-Verb
0.6329
0.6202
0.5737
g12_camperplus
g13_planningpoker
g14_datahub
TF-IDF+K-means
0.5774
0.7665
0.6718
TF-IDF+DBSCAN
0.2823
0.4068
0.3946
TF-IDF+Spectral
0.6526
0.6658
0.5724
Doc2Vec+K-means
0.3456
0.6548
0.3711
Doc2Vec+DBSCAN
0.0831
0.0951
0.2955
Doc2Vec+Spectral
0.5585
0.5484
0.2791
TCNN+K-means
0.4127
0.6976
0.5650
TCNN+DBSCAN
0.3106
0.3978
0.4508
TCNN+Spectral
0.3882
0.6166
0.5244
CUSNLP-Noun
0.6379
0.3740
0.4614
CUSNLP-Verb
0.6992
0.6604
0.6120
Tab.7
Fig.7
Fig.8
1
A, Sillitti G Succi . Requirements engineering for agile methods. In: Aurum A, Wohlin C, eds. Engineering and Managing Software Requirements. Berlin, Heidelberg: Springer, 2005, 309−326
2
Leffingwell D. Agile Software Requirements: Lean Requirements Practices for Teams, Programs, and the Enterprise. Upper Saddle River: Addison-Wesley Professional, 2011
3
X, Wang L, Zhao Y, Wang J Sun . The role of requirements engineering practices in agile development: an empirical study. In: Zowghi D, Jin Z, eds. Requirements Engineering. Berlin, Heidelberg: Springer, 2014, 195−209
4
Kassab M. The changing landscape of requirements engineering practices over the past decade. In: Proceedings of the 5th IEEE International Workshop on Empirical Requirements Engineering (EmpiRE). 2015, 1−8
5
S, Dimitrijević J, Jovanović V Devedžić . A comparative study of software tools for user story management. Information and Software Technology, 2015, 57: 352–368
6
J, Patton P Economy . User Story Mapping: Discover the Whole Story, Build the Right Product. Sebastopol: O’Reilly Media, Inc., 2014
7
C H, Wang Z, Jin H Y, Zhao L, Liu W, Zhang M Y Cui . Human-assisted elicitation and evolution of user stories with scenarios. Journal of Software, 2019, 30( 10): 3186–3205
8
G, Lucassen F, Dalpiaz der Werf J M E M, van S Brinkkemper . Visualizing user story requirements at multiple granularity levels via semantic relatedness. In: Proceedings of the 35th International Conference on Conceptual Modeling. 2016, 463−478
9
Wautelet Y, Heng S, Kolp M, Mirbel I, Poelmans S. Building a rationale diagram for evaluating user story sets. In: Proceedings of the 10th IEEE International Conference on Research Challenges in Information Science (RCIS). 2016, 1–12
10
K, Tsilionis J, Maene S, Heng Y, Wautelet S Poelmans . Conceptual modeling versus user story mapping: which is the best approach to agile requirements engineering? In: Proceedings of the 15th International Conference on Research Challenges in Information Science. 2021, 356–373
11
J, Berends F Dalpiaz . Refining user stories via example mapping: an empirical investigation. Proceedings of the 29th IEEE International Requirements Engineering Conference (RE), 2021, 345–355
12
Y, Wautelet S, Heng M, Kolp I Mirbel . Unifying and extending user story models. In: Proceedings of the 26th International Conference on Advanced Information Systems Engineering. 2014, 211−225
13
G, Grau X, Franch E, Mayol C, Ayala C, Cares M, Haya F, Navarrete P, Botella C Quer . RiSD: a methodology for building i* strategic dependency models. In: Proceedings of the 17th International Conference on Software Engineering and Knowledge Engineering. 2005, 259−266
14
Lucassen G, Dalpiaz F, van der Werf J M E M, Brinkkemper S. Forging high-quality user stories: towards a discipline for agile requirements. In: Proceedings of the 23rd IEEE International Requirements Engineering Conference (RE). 2015, 126−135
15
Kanungo T, Mount D M, Netanyahu N S, Piatko C D, Silverman R, Wu A Y. An efficient k-means clustering algorithm: analysis and implementation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24: 881
16
Ester M, Kriegel H P, Sander J, Xu X. A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining. 1996, 226−231
17
M, Belkin P Niyogi . Laplacian Eigenmaps and spectral techniques for embedding and clustering. In: Proceedings of the 14th International Conference on Neural Information Processing Systems: Natural and Synthetic. 2001, 585−591
18
Joachims T. A probabilistic analysis of the rocchio algorithm with TFIDF for text categorization. In: Proceedings of the 14th International Conference on Machine Learning. 1997, 143–151
19
Q, Le T Mikolov . Distributed representations of sentences and documents. In: Proceedings of the 31st International Conference on Machine Learning. 2014, 1188−1196
20
J, Xu P, Wang G, Tian B, Xu J, Zhao F, Wang H Hao . Short text clustering via convolutional neural networks. In: Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing. 2015, 62−69
21
C, Goutte E Gaussier . A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. In: Proceedings of the 27th European Conference on Information Retrieval. 2005, 345−359
22
Larsen B, Aone C. Fast and effective text mining using linear-time document clustering. In: Proceedings of the 5th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1999, 16–22
23
Hedges L V, Olkin I. Statistical Methods for Meta-Analysis. New York: Academic Press, 1985
24
S S Sawilowsky . New effect size rules of thumb. Journal of Modern Applied Statistical Methods, 2009, 8( 2): 597–599
25
Rodeghero P, Jiang S, Armaly A, McMillan C. Detecting user story information in developer-client conversations to generate extractive summaries. In: Proceedings of the 39th IEEE/ACM International Conference on Software Engineering (ICSE). 2017, 49−59
26
G, Lucassen F, Dalpiaz der Werf J M E M, van S Brinkkemper . Improving agile requirements: the Quality User Story framework and tool. Requirements Engineering, 2016, 21( 3): 383–403
27
Robeer M, Lucassen G, van der Werf J M E M, Dalpiaz F, Brinkkemper S. Automated extraction of conceptual models from user stories via NLP. In: Proceedings of the 24th IEEE International Requirements Engineering Conference (RE). 2016, 196−205
28
F, Dalpiaz der Schalk I, van G Lucassen . Pinpointing ambiguity and incompleteness in requirements engineering via information visualization and NLP. In: Proceedings of the 24th International Working Conference on Requirements Engineering: Foundation for Software Quality. 2018, 119−135
29
Y, Wautelet S, Heng D, Hintea M, Kolp S Poelmans . Bridging user story sets with the use case model. In: Proceedings of 2016 International Conference on Conceptual Modeling. 2016, 127−138
30
Mesquita R, Jaqueira A, Agra C, Lucena M, Alencar F. US2StarTool: generating i* models from user stories. In: Proceedings of the 8th International i* Workshop (istar 2015). 2015, 103−108
31
A, Jaqueira M, Lucena F M R, Alencar J, Castro E Aranha . Using i* models to enrich user stories. In: Proceedings of the 6th International i* Workshop 2013. 2013, 55−60
32
M, Trkman J, Mendling M Krisper . Using business process models to better understand the dependencies among user stories. Information and Software Technology, 2016, 71: 58–76
33
Y, Wautelet M, Velghe S, Heng S, Poelmans M Kolp . On modelers ability to build a visual diagram from a user story set: a goal-oriented approach. In: Proceedings of the 24th International Working Conference on Requirements Engineering: Foundation for Software Quality. 2018, 209−226
34
Barbosa R, Silva A E A, Moraes R. Use of similarity measure to suggest the existence of duplicate user stories in the srum process. In: Proceedings of the 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshop (DSN-W). 2016, 2−5
35
C, Li Y, Duan H, Wang Z, Zhang A, Sun Z Ma . Enhancing topic modeling for short texts with auxiliary word embeddings. ACM Transactions on Information Systems, 2017, 36( 2): 11
36
X, Quan C, Kit Y, Ge S J Pan . Short and sparse text topic modeling via self-aggregation. In: Proceedings of the 24th International Conference on Artificial Intelligence. 2015, 2270−2276
37
S, Seifzadeh A K, Farahat M S, Kamel F Karray . Short-text clustering using statistical semantics. In: Proceedings of the 24th International Conference on World Wide Web. 2015, 805−810
38
A, Joulin E, Grave P, Bojanowski T Mikolov . Bag of tricks for efficient text classification. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. 2017, 427−431
39
Boom C, De Canneyt S, Van S, Bohez T, Demeester B Dhoedt . Learning semantic similarity for very short texts. In: Proceedings of 2015 IEEE International Conference on Data Mining Workshop (ICDMW), 2015, 1229−1234
40
J, Zeng J, Li Y, Song C, Gao M R, Lyu I King . Topic memory networks for short text classification. In: Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. 2018, 3120−3131
41
T, Kenter Rijke M de . Short text similarity with word embeddings. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. 2015, 1411−1420
42
Hua W, Wang Z, Wang H, Zheng K, Zhou X. Short text understanding through lexical-semantic analysis. In: Proceedings of the 31st IEEE International Conference on Data Engineering. 2015, 495−506
43
S, Liang E, Yilmaz E Kanoulas . Dynamic clustering of streaming short documents. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016, 995−1004
44
Y, Zuo J, Wu H, Zhang H, Lin F, Wang K, Xu H Xiong . Topic modeling of short texts: a pseudo-document view. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016, 2105−2114
45
J, Yin D, Chao Z, Liu W, Zhang X, Yu J Wang . Model-based clustering of short text streams. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018, 2634−2642
46
Banerjee S, Ramanathan K, Gupta A. Clustering texts using wikipedia. In: Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval. 2007
47
S, Fodeh B, Punch P N Tan . On ontology-driven document clustering using core semantic features. Knowledge and Information Systems, 2011, 28( 2): 395–421
48
J, Yin J Wang . A Dirichlet multinomial mixture model-based approach for short text clustering. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2014, 233−242
49
Lai S, Xu L, Liu K, Zhao J. Recurrent convolutional neural networks for text classification. In: Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2015, 2267−2273
50
Ravi S, Kozareva Z. Self-governing neural networks for on-device short text classification. In: Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. 2018, 804−810
51
J, Xie R, Girshick A Farhadi . Unsupervised deep embedding for clustering analysis. In: Proceedings of the 33rd International Conference on International Conference on Machine Learning. 2016, 478−487
52
X, Zhang J, Zhao Y LeCun . Character-level convolutional networks for text classification. In: Proceedings of the 28th International Conference on Neural Information Processing Systems. 2015, 649−657
53
M, Wang Z, Lu H, Li Q Liu . Syntax-based deep matching of short texts. In: Proceedings of the 24th International Conference on Artificial Intelligence. 2015, 1354−1361
54
P, Wang J, Xu B, Xu C L, Liu H, Zhang F, Wang H Hao . Semantic clustering and convolutional neural network for short text categorization. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015, 352−357
