Towards estimating computer users’ mood from interaction behaviour with keyboard and mouse

doi:10.1007/s11704-013-2331-z

Front. Comput. Sci.

2013, Vol. 7

Issue (6) : 943-954 https://doi.org/10.1007/s11704-013-2331-z

RESEARCH ARTICLE

Towards estimating computer users’ mood from interaction behaviour with keyboard and mouse

Iftikhar Ahmed KHAN¹(

), Willem-Paul BRINKMAN², Robert HIERONS³

¹. Department of Computer Software Engineering, University of Engineering & Technology, Mardan 23200, Pakistan
². Interactive Intelligence Group, Delft University of Technology, Delft 2628B, the Netherlands
³. Department of Information Systems and Computing, Brunel University, West London UB83PH, UK

Download: PDF(0 KB)
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

The purpose of this exploratory research was to study the relationship between the mood of computer users and their use of keyboard and mouse to examine the possibility of creating a generic or individualized mood measure. To examine this, a field study (n = 26) and a controlled study (n = 16) were conducted. In the field study, interaction data and self-reported mood measurements were collected during normal PC use over several days. In the controlled study, participants worked on a programming task while listening to high or low arousing background music. Besides subjective mood measurement, galvanic skin response (GSR) data was also collected. Results found no generic relationship between the interaction data and the mood data. However, the results of the studies found significant average correlations between mood measurement and personalized regression models based on keyboard and mouse interaction data. Together the results suggest that individualized mood prediction is possible from interaction behaviour with keyboard and mouse.

Keywords keyboard mouse interaction mood measure computer users programming

Corresponding Author(s): Iftikhar Ahmed KHAN

Issue Date: 01 December 2013

Cite this article:

Iftikhar Ahmed KHAN,Willem-Paul BRINKMAN,Robert HIERONS. Towards estimating computer users’ mood from interaction behaviour with keyboard and mouse[J]. Front. Comput. Sci., 2013, 7(6): 943-954.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-013-2331-z
https://academic.hep.com.cn/fcs/EN/Y2013/V7/I6/943

1	S Brave, C Nass. Emotion in human-computer interaction. The humancomputer interaction handbook: fundamentals, evolving technologies and emerging applications, 2002, 81−96
2	R Plutchik. Emotion: a psychoevolutionary synthesis. Harper & Row New York, 1980
3	R W Picard. Affective Computing. MIT Press, 2000
4	P Zimmermann, S Guttormsen, B Danuser, P Gomez. Affective computing—a rationale for measuring mood with mouse and keyboard. International Journal of Occupational Safety and Ergonomics, 2003, 9(4): 539−551
5	J T Klein. Computer response to user frustration. PhD thesis, Massachusetts Institute of Technology, 1998
6	J Lazar, A Jones, M Hackley, B Shneiderman. Severity and impact of computer user frustration: a comparison of student and workplace users. Interacting with Computers, 2006, 18(2): 187−207 https://doi.org/10.1016/j.intcom.2005.06.001
7	J M Ross, H Zhang. Structured programmers learning object-oriented programming: cognitive considerations. ACM SIGCHI Bulletin, 1997, 29(4): 93−99 https://doi.org/10.1145/270950.270999
8	I A Khan, W P Brinkman, R M Hierons. Domoods affect programmers’ debug performance? Cognition, Technology & Work, 2011, 13(4): 245−258 https://doi.org/10.1007/s10111-010-0164-1
9	P J Lang. The emotion probe: studies of motivation and attention. American Psychologist, 1995, 50(5): 372−385 https://doi.org/10.1037/0003-066X.50.5.372
10	T Buchanan, J A Johnson, L R Goldberg. Implementing a five-factor personality inventory for use on the internet. European Journal of Psychological Assessment, 2005, 21(2): 115−127 https://doi.org/10.1027/1015-5759.21.2.115
11	M Valstar, I Patras, M Pantic. Facial action unit recognition using temporal templates. In: Proceedings of the 13th IEEE International Workshop on Robot and Human Interactive Communication. 2004, 253−258
12	G H Gendolla. On the impact of mood on behavior: an integrative theory and a review. Review of General Psychology, 2000, 4(4): 378−408 https://doi.org/10.1037/1089-2680.4.4.378
13	J P Forgas, K Fiedler. Us and them: mood effects on intergroup discrimination. Journal of Personality and Social Psychology, 1996, 70(1): 28−40 https://doi.org/10.1037/0022-3514.70.1.28
14	G K Manucia, D J Baumann, R B Cialdini. Mood influences on helping: direct effects or side effects? Journal of Personality and Social Psychology, 1984, 46(2): 357−364 https://doi.org/10.1037/0022-3514.46.2.357
15	V C Ottati, L M Isbell. Effects on mood during exposure to target information on subsequently reported judgments: an on-line model of misattribution and correction. Journal of Personality and Social Psychology, 1996, 71(1): 39−53 https://doi.org/10.1037/0022-3514.71.1.39
16	A M Isen, S F Simmonds. The effect of feeling good on a helping task that is incompatible with good mood. Social Psychology, 1978, 346−349 https://doi.org/10.2307/3033588
17	A M Isen, N Geva. The influence of positive affect on acceptable level of risk: the person with a large canoe has a large worry. Organizational Behavior and Human Decision Processes, 1987, 39(2): 145−154 https://doi.org/10.1016/0749-5978(87)90034-3
18	T E Nygren, A M Isen, P J Taylor, J Dulin. The influence of positive affect on the decision rule in risk situations: focus on outcome (and especially avoidance of loss) rather than probability. Organizational Behavior and Human Decision Processes, 1996, 66(1): 59−72 https://doi.org/10.1006/obhd.1996.0038
19	M T Pham. Representativeness, relevance, and the use of feelings in decision making. Journal of Consumer Research, 1998, 25(2): 144−159 https://doi.org/10.1086/209532
20	N Johnson, A Galata, D Hogg. The acquisition and use of interaction behaviour models. In: Proceedings of the 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 1998, 866−871
21	D Gavrila, L Davis. Towards 3-d model-based tracking and recognition of human movement: a multi-view approach. In: Proceedings of the 1995 International Workshop on Automatic Face-and Gesture-Recognition. 1995, 272−277
22	I Lefter, L J Rothkrantz, P Wiggers, D A Van Leeuwen. Emotion recognition from speech by combining databases and fusion of classifiers. In: Proceedings of the 13th International Conference on Text, Speech and Dialogue. 2010, 353−360 https://doi.org/10.1007/978-3-642-15760-8_45
23	A F Bobick, J W Davis. Action recognition using temporal templates. Motion-Based Recognition, 1997, 9: 125−146
24	S Sakurazawa, N Yoshida, N Munekata, A Omi, H Takeshima, H Koto, K Gentsu, K Kimura, K Kawamura, M Miyamoto, R Arima, T Mori, T SekIya, T Furukawa, Y Hashimoto, H Numata, J I Akita, Y Tsukahara, H Matsubara. A computer game using galvanic skin response. In: Proceedings of the 2nd International Conference on Entertainment Computing, ICEC ’03. 2003, 1−3
25	W Boucsein, R W Backs. Engineering psychophysiology as a discipline: historical and theoretical aspects. Engineering Psychophysiology. Issues and Applications, 2000, 3−30
26	G Chanel, K Ansari-Asl, T Pun. Valence-arousal evaluation using physiological signals in an emotion recall paradigm. In: Proceedings of the 2007 IEEE International Conference on Systems, Man and Cybernetics. 2007, 2662−2667
27	W Boucsein, M Thum. Design of work/rest schedules for computer work based on psychophysiological recovery measures. International Journal of Industrial Ergonomics, 1997, 20(1): 51−57 https://doi.org/10.1016/S0169-8141(96)00031-5
28	E Haider, H Luczak, W Rohmert. Ergonomics investigations of workplaces in a police command-control centre equipped with tv displays. Applied Ergonomics, 1982, 13(3): 163−170 https://doi.org/10.1016/0003-6870(82)90001-1
29	L M Schleifer, R Ley. End-tidal PCO2 as an index of psychophysiological activity during VDT data-entry work and relaxation? Ergonomics, 1994, 37(2): 245−254 https://doi.org/10.1080/00140139408963642
30	C L Lisetti, F Nasoz. Maui: a multimodal affective user interface. In: Proceedings of the 10th ACM International Conference onMultimedia. 2002, 161−170
31	W Mähr, R Carlsson, J Fredriksson, O Maul, M Fjeld. Tabletop interaction: research alert. In: Proceedings of the 4th Nordic Conference on Human-Computer Interaction: Changing Roles. 2006, 499−500
32	P Khanna, M Sasikumar. Recognising emotions from keyboard stroke patterns. International Journal of Computer Applications, 2010, 11(9): 1−5
33	G A Tsihrintzis, M Virvou, E Alepis, I O Stathopoulou. Towards improving visual-facial emotion recognition through use of complementary keyboard-stroke pattern information. In: Proceedings of the 5th International Conference on Information Technology: New Generations. 2008, 32−37
34	J Russell. A circumplex model of affect. Journal of Personality and Social Psychology, 1980, 39(6): 1161−1178 https://doi.org/10.1037/h0077714
35	M M Bradley, P J Lang. Measuring emotion: the self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 1994, 25(1): 49−59 https://doi.org/10.1016/0005-7916(94)90063-9
36	A Mehrabian. Basic Dimensions for A General Psychological Theory. OG&H Publishers, 2008
37	A Mehrabian. Behavioural treatment and bio-behavioural assessment: computer applications. Oelgeschlager, Gunn & Hain, 1980
38	J Cohen. Statistical power analysis. Current Directions in Psychological Science, 1992, 1(3): 98−101 https://doi.org/10.1111/1467-8721.ep10768783
39	K Card, T Moran, A Newell. The Psychology of Human-Computer Inter action. Lawrence Erlbaum Publishers, 1983
40	M Borenstein, L V Hedges, J P Higgins, H R Rothstein. Introduction to meta-analysis. John Wiley & Sons, 2009 https://doi.org/10.1002/9780470743386
41	H Wang, H Prendinger, T Igarashi. Communicating emotions in online chat using physiological sensors and animated text. In: Proceedings of the 2004 Extended Abstracts on Human Factors in Computing Systems Conference. 2004, 1171−1174 https://doi.org/10.1145/985921.986016
42	P Shepherd. Tools for transformation. 2001
43	T Albinoni. Adagio in g minor for organ and strings, Perivale: England Warner Classics, 1996
44	W F Thompson, E G Schellenberg, G Husain. Arousal, mood, and the mozart effect. Psychological Science, 2001, 12(3): 248−251 https://doi.org/10.1111/1467-9280.00345
45	P J Silvia, A E Abele. Can positive affect induce self-focused attention? methodological and measurement issues. Cognition & Emotion, 2002, 16(6): 845−853 https://doi.org/10.1080/02699930143000671
46	H Moby. On everything is wrong. New York: Elektra, 1995
47	J Tukey. Exploratory data analysis. Addison-Wesley Press, 1977
48	R Baumgartner, L Ryner, R Somorjai, R Summers. Exploratory data analysis reveals spatio-temporal structure of null FMRI data. In: Proceedings of the 2000 International Society for Magnetic Resonance in Medicine. 2000, 1717
49	A Gaillard. Theoretical and methodological issues in psychophysiological research. Engineering Psychophysiology, 1998
50	R Harper. Being human: Human-computer interaction in the year 2020. Microsoft Research, 2008
51	J Robison, S McQuiggan, J Lester. Evaluating the consequences of affective feedback in intelligent tutoring systems. In: Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. 2009, 1−6
52	J Zhai, A Barreto. Stress detection in computer users through noninvasive monitoring of physiological signals. Biomedical Sciences Instrumentation, 2006, 42: 495−500
53	M Pantic, L J Rothkrantz. Toward an affect-sensitive multimodal human-computer interaction. Proceedings of the IEEE, 2003, 91(9): 1370−1390 https://doi.org/10.1109/JPROC.2003.817122
54	A G Chitu, L J Rothkrantz, P Wiggers, J C Wojdel. Comparison between different feature extraction techniques for audio-visual speech recognition. Journal on Multimodal User Interfaces, 2007, 1(1): 7−20 https://doi.org/10.1007/BF02884428

[1]	Qingyang LI, Zhiwen YU, Huang XU, Bin GUO. Human-machine interactive streaming anomaly detection by online self-adaptive forest[J]. Front. Comput. Sci., 2023, 17(2): 172317-.
[2]	Shiwei CHENG, Jialing WANG, Xiaoquan SHEN, Yijian CHEN, Anind DEY. Collaborative eye tracking based code review through real-time shared gaze visualization[J]. Front. Comput. Sci., 2022, 16(3): 163704-.
[3]	Xiangmao MENG, Wenkai LI, Xiaoqing PENG, Yaohang LI, Min LI. Protein interaction networks: centrality, modularity, dynamics, and applications[J]. Front. Comput. Sci., 2021, 15(6): 156902-.
[4]	Bin GUO, Yasan DING, Yueheng SUN, Shuai MA, Ke LI, Zhiwen YU. The mass, fake news, and cognition security[J]. Front. Comput. Sci., 2021, 15(3): 153806-.
[5]	Yao LU, Xinjun MAO, Tao WANG, Gang YIN, Zude LI. Improving students’ programming quality with the continuous inspection process: a social coding perspective[J]. Front. Comput. Sci., 2020, 14(5): 145205-.
[6]	Yanbin WANG, Zhuhong YOU, Liping LI, Zhanheng CHEN. A survey of current trends in computational predictions of protein-protein interactions[J]. Front. Comput. Sci., 2020, 14(4): 144901-.
[7]	Shaocheng GUO, Songcan CHEN, Qing TIAN. Ordinal factorization machine with hierarchical sparsity[J]. Front. Comput. Sci., 2020, 14(1): 67-83.
[8]	Zhibin YANG, Jean-Paul BODEVEIX, Mamoun FILALI. Towards a simple and safe Objective Caml compiling framework for the synchronous language SIGNAL[J]. Front. Comput. Sci., 2019, 13(4): 715-734.
[9]	Xuepeng FAN, Xiaofei LIAO, Hai JIN. FunctionFlow: coordinating parallel tasks[J]. Front. Comput. Sci., 2019, 13(1): 73-85.
[10]	Shan ZHONG, Quan LIU, Zongzhang ZHANG, Qiming FU. Efficient reinforcement learning in continuous state and action spaces with Dyna and policy approximation[J]. Front. Comput. Sci., 2019, 13(1): 106-126.
[11]	Wenting ZHAO, Lijin WANG, Yilong YIN, Bingqing WANG, Yuchun TANG. Sequential quadratic programming enhanced backtracking search algorithm[J]. Front. Comput. Sci., 2018, 12(2): 316-330.
[12]	Xuansong LI, Xianping TAO, Jian LU. Towards a programming framework for activity-oriented context-aware applications[J]. Front. Comput. Sci., 2017, 11(6): 987-1006.
[13]	Donggang CAO, Lianghuan KANG, Hanglong ZHAN, Hong MEI. Towards application-level elasticity on shared cluster: an actor-based approach[J]. Front. Comput. Sci., 2017, 11(5): 803-820.
[14]	Xiaobing WANG, Cong TIAN, Zhenhua DUAN, Liang ZHAO. MSVL: a typed language for temporal logic programming[J]. Front. Comput. Sci., 2017, 11(5): 762-785.
[15]	Tianyi WANG,Yang CHEN,Yi WANG,Bolun WANG,Gang WANG,Xing LI,Haitao ZHENG,Ben Y. ZHAO. The power of comments: fostering social interactions in microblog networks[J]. Front. Comput. Sci., 2016, 10(5): 889-907.

Viewed

Full text

Abstract

Cited

Shared

Discussed