The Development of Web-Based Emotion Detection System Using Keyboard Actions (EDS-KA)

— Emotions are one of the critical user’s experiences factors. Emotion is a subjective feeling and it is challenging to determine human emotions. Thus, there is a need to have a system that could automatically detect human emotions. Nowadays, most of the emotion detection systems are obtrusive, complicated and expensive. Besides that, most of the data collection and analysis process for research has been done manually. Therefore, Web-based Emotion Detection System using Keyboards Actions (EDS-KA) has been developed to detect human emotions without their awareness, uncomplicated and inexpensive in which it is based on human actions using the computer keyboard. Besides, this system could assist researchers in data collection and analysis, whereas data could be collected and analyzed automatically. Further, the analysis results were saved in the database and could be printed out. EDS-KA adopted the Rational Unified Process (RUP) system development methodology. The RUP has been selected for EDS-KA development because it is an iterative software development process. Therefore, it enables produce well-defined system requirements and reduced the software risk. EDS-KA was a web-based system. Thus, users could access the system at any time and place that have an internet connection. In terms of emotional considerations, EDS-KA focuses on five (5) primary emotional states, namely happy, sad, afraid, relaxed and neutral (emotionless). Taken together, this system could assist the researchers’ tasks in conducting emotion detection research more efficient, taking less time and inexpensive.


I. INTRODUCTION
Emotion is a subjective experience; thus, it is difficult to recognize human emotion. Nowadays, numerous emotion detection systems can be used to detect a human's emotion. However, most of these emotion detection systems are obtrusive, complicated and expensive such as using physiological approaches, including electroencephalogram (EEG) signal [1], brain waves [2] and others. Consequently, due to these complex emotion detection system, researchers used approaches that involve low cost and does not involve expert skills to use the instruments such the Self-Assessment Manikin (SAM) rating affective pictures [3]. The SAM pictures could be offered to the respondents in the paper and digital format. Although using a computerized format to present the SAM picture to the participants, the data collection process is still done manually.
Therefore, to address this problem, a system which is namely Web-based Emotion Detection System using Keyboard Actions (EDS-KA) has been developed in order to detect the human emotion in unobtrusive and without user awareness which is based on the keyboard actions. Besides, EDS-KA was developed to solve the problem of manual data collection. EDS-KA would automatically detect and save the users' emotion data in the database. Further, it enables us to analyze the data and generate reports. As a result, researchers' tasks, which are data collection and analysis, become more comfortable, efficient, and saves time. Besides that, the data that are automatically collected by EDS-KA could prevent inaccurate data that input manually into the system by a human. EDS-KA has been developed on a webbased platform. EDS-KA only focuses on five (5) primary emotional states, which are happy, sad, afraid, relaxed and neutral (emotionless state).
The rest of the paper is organized as follows. The background of the study and literature review is discussed in Section I. Afterward, the methodology adopted for EDS-KA development is explained in Section II. Section III describes details of EDS-KA such as the system scopes, modules, architecture and so forth. Subsequently, Section IV describes the system benefits, limitations, and suggestions for future works. Finally, this paper concludes with a conclusion in Section IV.

A. The Background of the Study
In regards to getting a clear comprehension of the study area, this section describes in-depth about the emotions and keystroke, such as definitions of the terminology used, the theory of emotions, and the association between emotion and keystroke.

1) Definitions of Terminology
This section describes the terms used in this study in order to give a better understanding to the readers of this paper. Therefore, the operational definitions of the constructs used in this study, as provided as follows: a. Emotion: The terms "affect," "emotion," and "mood" are regularly utilized interchangeably, which leads to troubles when comparing different research results and methods. There is a massive definition of emotion in the literature. Parkinson (1995) defined emotion as a relatively short-term, evaluative state focused on a particular intentional object (a person, an event, or state of affairs) [4]. Short-term emotions in the learning environment are confusion, frustration, interest, boredom or happiness [5]. Affect is the term used to refer to both emotions and moods [6]. Emotion has the possessions of response: it usually has a specific cause, stimulus or other thought. It is commonly an intense experience of short duration (seconds to minutes), and the person usually is well aware of its event. [7].
In contrast, a mood seems to be not obvious, longerlasting, less thorough, more in the background, that encourages a person to move in the positive or negative affective state [7]. Besides, moods seem to be non-specific compared to emotions that are commonly precise (for example, happiness, anger, and many others) and emphasized an identifiable person, object or event [7]. In contrary to emotions, people might not be notified of their mood until their attention is drawn to it. Thus, from the definitions discussed, it seems that the terms "emotion" and "affect" are appropriate to apply in this study, and both terms are used interchangeably.
b. Keystroke: Keystroke is defined as a key on a keyboard, especially as a measure of work [8].

2) Theory of Emotion
Emotions can be conceptualized into a dimension or discrete emotion model. In discrete emotion theory, many emotions can depend on core emotions such as happiness, surprise, sadness, anger, disgust, and fear. On the other hand, emotion can be determined in dimensionally based on valence and arousal. Valence is defined as the point to which individuals use the pleasantness or unpleasantness indicated by affect words when identifying their subjective emotional states. [9]. On the other hand, arousal is a point to which individuals use the level of subjective arousal indicated by affect words when identifying their subjective emotional states [9]. Thus, it can be summarized that valence and arousal are the main parameters that could be used to determine the emotional state of the human. Fig. 1 shows a two-dimensional representation of an emotional state determined by valence and arousal.

3) The Association between Keystroke and Emotion
Khanna and Sasikumar [10] found that approximately 70% of the negative emotional user has low typing speed compared to a neutral state, whereas approximately 83% of positive emotional user have high typing speed compared to neutral state [10]. Thus, it can be concluded that humans with positive emotions will type faster, while negative emotions will type slower. In other words, human typing speed are related to valence.
Besides, this study found that keystroke duration is more responsive to valence, in which the accuracy rate is more responsive to arousal. Furthermore, the keystroke latency is influenced by both valence and arousal, with these two variables interacts with each other [10]. Thus, it can be concluded that the typing error rate is related to arousal.

B. Literature Review
This section discusses the literature review related to the emotion measurement/detection approaches. Humans show their emotions in various ways, such as through faces and voices, or actions such as mouse clicking and keyboardstroke patterns. Thus, a variety of approaches are needed to evaluate the human emotions that stimulated in various ways. Extensive research has been done in the emotion measurement domain. In general, Zimmermann et. al. classified the measurements of effects into three fields as follows [7]: • Physiological approaches, such as skin conductance, blood pressure, pupillary dilation, and brain waves. • Behavioral approaches such as facial expressions, voice intonation, gestures, and keyboard and mouse usage. • Psychological approaches such as self-reports (verbal explanations of an emotional state, rating scales, and graphical differentials methods). The physiological approach and its non-verbal equipment have been applied in several previous studies such as Li, Xu and Feng [11], Balters and Steinert [12], Zhao et al. [13]. The main benefits of this approach are as follows: • It is suitable to apply in various cultures since its nonverbal measurement is language independent [14]. • Data is gained concurrently (in real-time) with the task being performed [7]. • the measurement is unobtrusive in which the respondents will not be distracted during the measurement [15]. However, the physiological method also has a few weaknesses, as follows: • it is costly, difficult to use in which cables and electrodes attached directly to the people [1]. That equipment limits people from moving and acting easily. This situation might cause different emotions in people and therefore alter the results [16]. • The technical expertise is required to install the instruments and sensors [8]. Thus, this method is suitable for laboratory experiments and not for broader use since it involves high-cost equipment and expert skills to attach the equipment to the human body. Facial expressions have been generally applied for emotion recognition and have achieved promising results [11]. Nevertheless, this method is tested absolutely on "created" emotions expressions, for example from actors, rather than on real emotions [7]. Thus, it is predicted that emotion recognition accuracy would decrease seriously in natural situations. Moreover, it usually requires specialized hardware to be applied [17] such as a video camera in which respondents believe recording devices such as video cameras are obtrusive. People will typically act differently once they notice that they are being observed. Subsequently, fake emotions will be produced. In the other study, Samara et al [16] found that facial expressions cannot accurately expose the real emotions of users while interacting with ordinary computerized tasks.
A behavioral approach such as a keyboard and mouse usage had been applied in the previous studies by Kolakowska [17], Valdez et. al. [18], Vea and Rodrigo [15], Yamauchi and Xiao [19], Nassr et. al. [20]. This approach is an unobtrusive, non-invasive measurement method which is evaluating user affect concurrently with task processing. Moreover, it is low-cost in which it only uses standard computer input devices and no additional tools, such as a sensor or Web camera, are necessary to capture the respondents' emotions. According to Kolakowska [17], experiments have proved that keystroke characteristics could affect emotional states.
Self-reports are extensively used and still serve as the primary method for identifying emotions. Respondents need to state their emotions using a set of rating scales or verbal procedures [21]. The benefits of this approach are that it is low-cost and expert skills are not involved in using the instruments [22]. However, verbal instruments are hard to apply across societies. Furthermore, studies denote that the emotions which are stimulated from products are challenging to verbalize [23]. In regards to solving this difficulty, uncomplicated non-verbal self-report instruments have been created which use pictures rather than words to express emotional reactions.
The Self-Assessment Manikin (SAM) rating of affective pictures developed by Lang (1980) is based on a series of pictures of the puppets (graphical manikins) which are used for measuring dimensional emotions [24]. The dimensional emotion consists of valence, arousal and dominance. The SAM pictures are commonly offered to the respondents in the paper and digital format. The respondents are required to select the picture that represents their emotions by marking the corresponding picture on each emotion. Bargas-Avila and Hornbaek in their research on user experience discovered that the most common tool for measuring emotions is the SAM scale [25]. The benefits of the SAM scale are that it is well validated, a non-verbal approach that measures people's emotion experience in a fast manner and it is relevant in between-culture studies [25]; however, the constraints SAM emotional pictures which could be considered as a psychological approach is that participants might not report their accurate feelings since the past emotions are not simultaneous with the task performed [7].
As a result, it appears from the investigations above that emotion detection based on keystroke is still a fascinating research subject in the area of behavioral measurement of emotion due to its non-intrusiveness and handiness [21].

II. MATERIALS AND METHOD
This section discusses the materials and adopted methodology to develop a Web-based Emotion Detection System using Keyboard Actions (EDS-KA).

A. Materials
Programming languages are the materials that essentially required to develop a system. The core programming language used in the EDS-KA development was the Java programming language. Besides that, EDS-KA was a webbased platform. Hence, the web programming language such as HyperText Markup Language (HTML5), Java Saver Pages (JSP), JavaScript and JQuery had been used as well. In addition, the database for EDS-KA will be developed by using Structured Query Language (SQL).

B. Method
The Rational Unified Process (RUP) has been adopted as a system development methodology to develop EDS-KA. The RUP is an iterative software development process that uses an object-oriented and Web-enabled program development approach [26]. Since the RUP has been designed in conjunction with Unified Modeling Language (UML), so the workflows in the RUP description are oriented around associated UML models. The RUP has been selected for EDS-KA development because it is an iterative software development process that iteratively develops the software. Thus, it will produce precise system requirements. Besides, the software risk will be reduced due to the iterative approach in RUP. RUP consists of four (4) phases which are Inception, Elaboration, Construction and Transition phases. The details of each RUP phase relating to the EDS-KA development are discussed in the following subsections.

1) Inception Phase:
At this phase, all external entities (people and systems) that will interact with the system is defined. Then, the system developers use this information to evaluate the benefits of the development of the system. If the benefits are minimal, then the project will not have proceeded. A proposal for the EDS-KA had been produced at this phase, aims to convince the stakeholder to agree to the implementation of the project. This proposal includes an overview of the project, project goals, project scope and project objective. Besides that, a Software Project Management Plan (SPMP) document also had been produced to ensure all the activities that must be carried out starting from the beginning to the end of the project were wellplanned. Finally, the risk and contingencies also had been determined to estimate the potential risks.

2) Elaboration Phase:
This phase is the most critical of the four phases. At this phase, a Software Requirement Specifications (SRS) document had been produced to capture all the requirements needed to develop EDS-KA. Besides that, a Software Design Description (SDD) document had been produced to describe the architecture and design of the EDS-KA. In the SDD document, the system developer used the UML model to visualize the system architectural model in a diagram. The designed diagram consists of a case diagram usage, activity diagrams, class diagram, sequence diagrams and Entity-Relationship Diagram (ERD). Besides, the interfaces of EDs-KA also had been designed at this phase.

3) Construction Phase:
The construction phase is basically focused on system design, programming and testing. In this phase, the EDS-KA is constructed using the programming languages as mentioned in the Materials section. Furthermore, EDS-KA will be tested to confirm that the system meets the requirement as stated in SRS and to ensure the developed system will function properly as expected without error.

4) Transition
Phase: This is the last phase of the RUP and it is focused on transforming the system from the development team to the users and ensures that the system is working well in an actual environment.

C. EDS-KA Components
This section explains the details of EDS-KA in terms of scopes, emotions classification and keyboard action formula, system modules, system architecture, Entity Relationship Diagram (ERD) and system implementation.

1) The System Scopes
EDS-KA covers the following scopes: • EDS-KA considers five (5) basic emotional states, which are happy, sad, afraid, and neutral (emotionless). • EDS-KA allows the alphabet (A-Z/a-z) key, numeric (0-9) key, space key and backspace key for typing the input. • EDS-KA classified the emotions based on the typing speed and the frequency of errors.

2) Emotions Consideration
According to the theories of emotion, if the valence and arousal are known, then the emotional state of the human can be determined [8]. Fig. 2 shows the emotion classification in EDS-KA. As can be seen in Fig. 2, there are two parameters, which is the typing speed (refer as the x-axis) and the frequency of error (refer as y-axis) that used to categorize the happy, sad, relaxed and afraid emotions as well as the neutral (emotionless). The typing speed was divided into three (3) categories, which are fast (35WPM and above), standard (30WPM to 34WPM) and slow (29WPM and below). Besides that, the frequency of error is also divided into three (3) categories, which are high (68% and above), average (34% to 67%) and low (33% and below). Thus, the emotion classification in EDS-KA is based on the coordinate value of the typing speed and the frequency of error to determine the emotions.
The calculation for the typing speed and the typing error rate that applied in EDS-KA were based on the Shukla & Solanki (2013) study, which as the following formulas [27].
where, summation use of numeric, alphabet, whitespace and backspace key divide by five is considered as a word(s).

3) The System
Modules EDS-KA consists of two (2) main modules which are user (participant) and researcher (administrator) modules. Fig.  3shows the activity diagram of the user module. As can be seen in Fig. 3, initially, the participants sign in into EDS-KA. Once the username and password were verified, the participants are directed to the Learning Module (LM) Topic. At this stage, participants were required to read the passages provided to trigger the participant's emotions. Since this study examined the human's feelings, the passages were determined as emotionally neutral that contained the facts regarding certain topics. Then, the participants are required to answer the questions based on the passages that they have read. The questions provided are structure questions in order to participants typing the answer using a keyboard key. Besides that, the purpose of the provided questions was to assess the participants' performance (cognition). Afterward, the participants are required to give their real emotions after they submitted their answers. The purpose of this activity is to compare the real emotions and the detected emotions by the system. Finally, the participants could view their detected emotions and performance result. The performance result is an additional feature in EDS-KA in order to investigate the association between human emotions and cognitions. Fig. 4 shows the activity diagram of the researcher module. The activity starts when a researcher visits the learning module page. There are three (3) operations that can be performed by a researcher in order to manage the topic in the learning module, which is added, edit and delete the passages. Firstly, to add the passages, a researcher had to click the add topic button on the learning module page. Then, a researcher was required to input all the data needed to create a new passage. When a researcher click submits button, a new passage (topic) was created. Secondly, to edit the passages, a researcher had to click the edit button on a passage topic that he/she needs to edit. Afterward, he/she can change the content of the passages and click on the submit button after editing was done. Then, the edited topic will be updated. Finally, to perform delete operation, a researcher had to click the delete button on a passage topic that he/she needs to delete. Then, a confirmation box appeared, and he/she had to respond to a confirmation box whether the need to delete or not. The selected passage will be deleted if he/she had confirmed to delete it. analysis results can be downloaded as report files or it could be printed out. There were six (6) analysis results that a researcher can be viewed which were the detected emotions, the expectation emotions, comparison between emotions and keyboard actions, comparison between emotions and performance scores, comparison between emotions and performance time, and summarization report on research participants.

4) System Architecture
Model-View-Controller (MVC) architecture is the system architecture of EDS-KA. MVC architecture is also known as Java Server Pages (JSP) Model 2 architecture that used as a design model to develop a web application. It was used as a software architectural pattern for implementing user interfaces. The overall view of this architecture is as shown in Fig. 6. 1) Model: The model was used to manage the behavior and data of the application domain. Furthermore, it will follow the instruction to transform the state from the controller. In a web application, the model included the database tables, the session information and the rules for administering the transactions.

2) View:
A view is the representation of the model. It renders the model into a user interface for user interaction purposes. A single model can be used to present multiple views for different purposes. In a web application, a view contains HTML, CSS and server-side templates.

3) Controller:
The controller was used to convert the user interactions with the view into actions that the model will implement. It act as an interface between view and model. Basically, it receives the message from view and sends this message to model. In web application, the controller consists of client-side scripting, http request processing and business logic or pre-processing.

5) Entity-Relationship Diagram
Entity-relationship diagram (ERD) is a data modeling approach that graphically depicts the relationship between database entities. A type of conceptual schema or semantic data model of a system often called a relational database is the outcomes from the ERD. Fig. 7 shows the ERD of EDS-KA that consists of six (6)

A. Results
The result of this study is a developed EDSKA. The completed EDS-KA consists of two (2) modules which are the participant module and the administrator module. These two (2) modules are explained with the aid of the figures. The explanation begins with the Participant Module, then followed by an Administrator module consecutively. Fig. 8 shows the home page of EDSKA, which is the sign-in page for the participant. When the participants had successfully sign-in into the system, the instruction of the system flow and guidelines will pop-up as shown in Fig. 9.  Here, the participants can select any provided passages (topics) that they desired to read by clicking the "start attempt' button. Then, the participants were directed to the Passage and Questions interface as shown in Fig. 11.  However, if the participant did not agree with the detected emotion, they should click the "No" button. Subsequently, a pop-up message that asking their real emotion will be displayed as shown in Fig. 13. The purpose of this message is to confirm that the participants' emotion, which is detected based on keyboard typing and speed while answering the questions, is like the participants' real emotion. If the participants agree with the detected emotion by the system, they should click the "Yes" button and they will be directed to the participants' performance results interface as shown in Fig. 14. Next, they were required to select which emotion (from five emotions provided) that represent their real feeling. Later, participants' performance results (individual results) interface will be displayed once they had selected their real emotion, as shown in Fig. 15. Fig. 15 shows the participants' performance results (individual results) interface. Here, the participants were able to use marks for each question in the test topic. Besides that, they also were able to view their overall performance scores and the emotions detected by the system. The following is the details explanation about the Researcher (administrator) module. At sub-module My Profile, the researcher was able to view and update his/her profile content. While, at sub-module Managing Research Assistant, the researcher was able to view, add, block or unblock the research assistants involved in this project.
At the sub-module Learning Module, the researcher was able to add, view, edit or delete passages and questions from the learning module. Besides that, he/she was able to view the passage topic listed by sorting and searching the topics using a keyword or published status. Besides, he/she can adjust the number of topic shows per page.  Fig. 16 illustrates the Participants' Record sub-module, in which this interface showed the record of participants that had used this system. The participants' record consists of participant identification, gender, age in range, education level, number of performance record and registered date. The researcher was able to view and sort the participants' list by record items such as gender. Moreover, he/she was also able to adjust the number of participants show per page. Besides, he/she able to export the participant's list into the Excel file. At sub-module Analysis Results, a researcher was able to view the results of analysis. Fig. 17 shows the overall emotions and performance results interface for the selected participant (an individual record). Fig. 18 shows an actual participants' emotions results in the interface based on the keyboard actions, which are typing speed and typing error. These results were also displayed in graph format as shown in Fig. 19.  To the readers' knowledge, as mentioned in the Introduction section, this paper is an emphasis on the development of EDS-KA, in which, how EDS-KA was designed and developed by adopting RUP methodology. In other words, this paper did not focus on the evaluation or experiment part which involves participants, evaluations and so forth.
B. Discussion 1) System Benefits: EDS-KA was developed in order to assist the researchers in detecting human emotions. The details benefit of the EDS-KA to the researchers are explained as follows: • EDS-KA enables to automatically detect human emotions based on keystroke input by the participants. Thus, the emotions' data become more accurate because it could prevent errors in input data by a human. • EDS-KA enables human emotions detected by standard computer keyboard, in which this approach could detect human emotion unobtrusive and without user awareness. Thus, the emotions produced are real because the participants did not realize that they are being monitored. • EDS-KA enables human emotions detected at lowcost since the standard computer keyboard is not expensive and no additional tools are necessary to capture human emotions. • EDS-KA enable to analyze the human emotions' data automatically. Thus, EDS-KA could assist and save researchers' time for data analysis. • EDS-KA was a Web-based platform. Thus, it can be accessed anywhere and anytime if it is connected to the internet.

2) System Limitations and Future Suggestions:
Several EDS-KA limitations could be improved in future project development. The limitations and future recommendations for EDS-KA are as follows: • EDS-KA only consider five (5) basic emotional states, which are happy, sad, afraid, and neutral (emotionless). Therefore, it would be interesting for future systems to consider more varieties of emotions to be detected. • The parameter implemented in EDS-KA for emotion detection is limited. Thus, it is suggested to add more parameters or rules in order to determine human emotion more accurately by using the computer keyboard. • Keyboard devise only may not enough to capture human emotion. For example, transaction logs consider of what users are doing, but not what they are thinking or feeling. Thus, it is suggested that future systems incorporate the behavioral psychological approach with the physiological approach (such as galvanic skin response) and psychological approaches such as self-reports to increase recognition accuracy of emotion and to produce a better representation of emotions in the investigation of affective response. • EDS-KA emotion detection function may make incapable of the browser that does not support JavaScript. Besides, the browser compatibility issues for the system's interface may arise since the old browser may not cover all the EDS-KA functionalities.

IV. CONCLUSION
As a conclusion, the ability of EDS-KA that could detect human emotions by keyboard actions could facilitate the researchers' task in the data-gathering phase and data analysis as well. These characteristics of the system could provide more precise data and robust study findings. Besides that, EDS-KA also investigates the relationship between human emotion and cognition. Thus, this system also could assist the researchers in discovering the relationship between emotions cognitions.