Evaluation of User Experience on Using the "MoLearn" Application in Learning Activities of High School Students

— The shift of paradigm in learning certainly impacts the demands of changing the ability of educators. Educators need to hone their material/scientific fields' abilities, and the methods and media used, especially those using information technology such as Hybrid Learning Model. Hybrid Learning Model is learning to provide content for learning models in various media. In order to support the Hybrid Learning Model, a mobile-based learning application was designed. It is a mobile-based software application to support high school students' learning known as the MoLearn application. The development of the MoLearn application continues to be done by also taking into account the User Experience. Some applications for education were developed with no appropriate design. For this reason, it was necessary to evaluate the MoLearn application with an Evaluation of User Experience from the use of MoLearn application. This study measured using tools such as the User Experience Questionnaire. The results of the User Experience Questionnaire for evaluation showed that 200 respondents (high school students) from 6 different schools of East Java Province, Indonesia. Students gave good marks on attributes of attractiveness, clarity, efficiency, and accuracy scales, while the attributes of stimulation and novelty are excellent. Students' perceptions of websites and Android-based application design are considered to meet the creative, innovative, interesting, easy, and informative attributes.


I. INTRODUCTION
In the 21 st century and the digital (the industrial revolution 4.0) era, education has an important role in producing Human Resources (HR) who have superior competencies needed in work. Meanwhile, the demands of the curriculum and the development in the industrial revolution era 4.0 require educational institutions to create innovation that benefits the 21st-century skills-based education world [1]- [4]. The 2013 curriculum requires 21st-century learning based on HOTs so that students have superior competencies with a variety of skills that are in line with the demands of the 21st century and the industrial revolution 4.0, including literacy, critical thinking, scientific creativity, collaboration, utilizing Information and Communication Technology, and problemsolving skills [1]- [3], [5], [6]. Technology has a fundamental role in learning Information and Communication Technology, and it has been pervasive in our lives, and students use technology daily [7].
The shift of paradigm in learning certainly impacts the demands of changing the ability of educators. Educators need to hone their abilities in material/scientific fields and the methods and media are used, especially those that use information technology. This is based on the fact that students today are students in the Generation Z era. The characteristics of Z-Generation [8]:  Comfortable and highly dependent on technology, this is due to Z-Generation growing up surrounded by technology,  Multitasking with diverse online products and sophisticated technological equipment, and appreciate the simplicity and interactive design,  Have higher social responsibility with a wealth of information that can be accessed online,  Always connected, communicating through social networks, cross-country and cross-culture indirectly affect the way of thinking and decision-making processes. Based on the characteristics of Generation Z, learning cannot be conduct conventionally. Educators cannot ignore Generation Z's need for learning models that adapt to their characteristics. Educators-centered learning, face-to-face learning media, paper assignments that have to be submitted, working on assignments must be at home or on campus, and educators can only be met through face-to-face. Those are not learning methods need by Generation Z students. Educators must begin to think of a learning model that can align themselves with the needs of students, who always follow the rapid development of the current gadget. Mobile technology is a great opportunity to provide a relevant and interesting learning experience [9]- [12]. However, this does not mean leaving the conventional method, i.e., face-to-face, because the touch of humanity is still very important. This concept is often referred to as Hybrid Learning. Hybrid Learning Model provides content for learning models in various media [13], [14].

A. Hybrid Learning Model
In order to support the Hybrid Learning Model, in 2017, a mobile-based learning application was designed. It is a mobile-based software application to support high school students' learning known as MoLearn [15]. The developed learning models in the MoLearn application are expected to provide several benefits, including alternative learning models, the availability of learning models that can bridge the students' HOTS t, and reference material in developing other learning models [16]. The application built can be used by teachers to provide exercises and enrichment through various sources that have been provided. While from the students' side the application can be used as a resource for learning with the guidance of teachers in the classroom and independent learning and enrichment. Blended learning can change the passive students who usually only receive knowledge, to be the active students who construct their knowledge [17]. Blended learning was found to be valid, practical, effective for learning, and can be used as a supplement to learning activities [18]. The development of the MoLearn application continues to be done by also considering the User Experience (UX).
There are several definitions of UX, one of which was delivered by Hassenzahl and Tractinsky, who defined UX as a technology that meets more than just user's needs. UX is a consequence of user's internal state, whether it is related to user's tendency, expectations, needs, motivation, mood etc., the characteristics of the system being designed (e.g., complexity, goals, usability, functions, etc.), and context (the environment in which interactions occur) [19]. UX is defined as a combination of product usability and design aspects [20]. Product usability includes simplicity and effectiveness in completing the task, while the design aspect focuses more on the elements of pleasing user, aesthetics, joy, and emotion generated. Hassenzahl presents the UX model from users' perspective by explaining the product elements in UX [21]. Situations related to the product's character from pragmatic and hedonic aspects are also consequences that users obtain.
One of MoLearn application users is a group of students, and the development of this application also concerns students directly. In 2019, researchers who are also application developers found it necessary to evaluate from user experience (UX). Masarweh said that some mobile learning applications were developed with no appropriate design [22]; thus, it was necessary to evaluate the MoLearn application. Evaluation of UX from the use of MoLearn application in this study is measured using tools such as the User Experience Questionnaire (UEQ). UEQ is widely used to measure user experience on a product, determine user ratings of products, and determine areas of improvement if needed [23]- [25].
In this study, measurements were performed on user experience level from 200 high school students, with the measurement instrument used was UEQ which has six scales with a total of 26 attributes categorized based on the measurement scales contained in UEQ [26]- [28]. The six scales in question are as follows:  Attractiveness: how attractive a product is as a whole perception.  Perspicuity: how easy a product to be used  Efficiency: how quickly users' complete tasks.  Dependability: can the user control the interaction?  Stimulation: how well a product motivates users.  Novelty: how innovative a product is.
Attractiveness is a pure valence dimension, not including any quality aspect. Perspicuity, efficiency, and dependability are included in the pragmatic quality aspects that are goaloriented so that users must perform the task of achieving the goal through the website. While stimulation and novelty are hedonic quality aspects that are not goal-oriented, users only access the website without the need to reach the goal. Fig. 1 below shows the structure of questions that fall into the measurement scale at UEQ.  Table 1 shows the measurement instrument of UEQ consisting of 26 coupled attributes that oppose one another. The purpose of this tool is to make UEQ data analysis as easy as possible by automating the calculation of all statistics needed to interpret the results. Several graphs are displayed to make it easier to understand the results of the MoLearn application measurement.

B. Method
This research, aiming to create a web and android-based learning application for the High School level, followed a gradual research development. It was utilized from the ADDIE model (analysis, design, development, implementation, and evaluation) [29] in the research stage.  [29] This study focuses on the evaluation stage. The application has been structured and has been used until the final stage is evaluation using a measuring instrument prepared. Several steps in the evaluation were carried out, starting from preparing the measured application and measuring instrument, determining the respondent, distributing measuring instruments, and processing data using UEQ.

C. Participant
The participants of this study were 200 high school students from 6 different schools of East Java Province and of 4 different cities, i.e., Surabaya, Sidoarjo, Jombang, and Nganjuk.

III. RESULT AND DISCUSSION
The evaluated application was the MoLearn application with three user groups: Administrator (MGMP), Teacher, and Student. Each has unique features according to needs in the learning process. The feature in question is explained in detail according to user groups.

A. Participants Administrator
Administrators' role in providing data and managing classes and serving as an administrator is MGMP (Consensus Points of Teacher Education). Here are menus that administrators can use:  Home: this page contains announcements in the form of moving banners to inform MGMP administrators, teachers, and students.  School Master: it is used to storing school information using the MoLearn application. Schools that are not listed on this menu will not be able to enter teacher and student data.  Teacher Master: this menu is used to registering teachers from a school that uses the MoLearn application. There are two ways to enter teacher data by entering one by one and importing it from excel 2003 (.xls) file.  Student Master: this menu is used to registering students from a school using the MoLearn application. There are two ways to enter student data by entering it one by one and importing it from Excel 2003 (.xls) files.  Regional Master; this menu lists regions ranging from provinces, cities, sub-regency to villages. This menu is used for consistency in using regional names in schools, teachers, and students.  School Year: it is used to regulate the school year that is currently active in the system, and all administrations are filtered according to the school year included in this menu.  Competency: this menu is used to create a learning framework from MGMP for teachers to provide materials, assignments, examinations under MGMP directives. There are three components with the highest level in this menu: standard competence, then the second is basic competence, and the last is indicators.
 MGMP material; this menu is used for MGMP administrators to provide materials for teachers or students. The material provided can be in the form of files (pdf, .doc, etc.), videos sourced from YouTube, and can be in the form of text or writing.  Teacher's Materials: this menu is used by administrators to monitor the materials provided by teachers at school to students.  Exam Questions: This menu is used for MGMP administrators to create question banks that teachers can later use as online exam questions through the MoLearn application.  Discussion Categories: it comprises the categories that are used for group discussions so that topics discussed in the MGMP teacher discussion menu are not mixed.  The announcement: this menu is used to make announcements via moving pictures that appear on the menu of the administrator, teacher, or student homepage.  Group User: this menu is used by super administrators (who have the right to access the highest level) to group the applications that can access any menu.  User: this menu is used to adding or changing user's data who can access the MoLearn application as an administrator.  Teacher Registration: this menu is used in order that MGMP can carry out the validation process for new users who already registered.

B. Teacher
The MoLearn application for teachers has function as a provider of local subjects or extras that will be given to students in their classrooms and create and organize assignments and examinations. Here are the menus that the teacher can access:  Home, this page has announcements in the form of moving banners to inform MGMP administrators, teachers, and students.  My Class, this menu is used by the teacher to arrange classes for one semester.  Competence, this menu is used by the teacher to see the list of standard competencies/core competencies and basic competencies of each core competency and indicators.  MGMP material, this menu is used by the teacher to download materials provided by the MGMP administrator.  Teacher's Material, this menu is used by the teacher to provide materials to students they teach. The material provided can be in the form of files (pdf, .doc, etc.), videos sourced from YouTube, and can be in the form of text or writing.  Create Assignments; the teacher uses this menu to assign assignments to be submitted through the MoLearn app.  Assignment's Score: this menu is used to view the score of assignments that have been submitted by students while assessing the works or assignments that have been submitted.  Create Exams; teacher uses this menu to create an exam schedule through the MoLearn application while the teacher chooses questions from the question bank that the MGMP administrators have made.  Exam Score: the teacher uses this menu to see the answers to online exams done by students through the MoLearn application.  Class Discussion: the teacher uses this menu to interact with students about the lesson or other things to support teaching and learning activities in class.  MGMP Teacher's Discussion; teachers and administrators use this menu.  Create Exam Questions; teachers can make their exam questions, which can be used in their class only.

C. Students
The MoLearn application for students has function as a place to download material, submit their assignments, and work on examinations and perform discussion with their classmates and teachers. The following are menus that students can access:  Home: this page contains announcements in the form of moving banners to inform MGMP administrators, teachers, and students.  MGMP Material; students use this menu to get materials given by MGMP administrators.  Teacher's Materials: students use this menu to get the materials given by their class teachers.  Class Assignments: students use this menu to view and submit assignments given by their teachers.  Exams, this menu is used by students to work on exams made by the teacher. After the exam is completed and assessed by the teacher, students can also see their test scores.  Class Discussion: Students use this menu to interact with their classmates and the teacher to talk about lessons or other things supporting teaching and learning activities in class.

D. The distribution of the User Experience Questionnaire
The various features available above, whether for groups of administrators, teachers, and also students. The special evaluation was conducted on the student group users on the use of the MoLearn application by distributing UEQ. The distribution of UEQ was carried out on 200 high school students from 6 schools from 4 different cities. The user experience scale calculation results in the questionnaire are shown in Table 3 below, namely: attractiveness, efficiency, perspicuity, dependability, stimulation, and novelty. Scores between -0.8 and 0.8 represent evaluations that are more or less than the corresponding scale, > 0.8 represent positive evaluation, and <-0.8 represent negative evaluation.
Thus, a pretty good score is +1.5 for the scale seen from a purely visual perspective in the scale range of -3 to +3. Fig. 3 below is the result of UX calculation. The results from UEQ calculations also classify results into pragmatic quality (Perspicuity, Efficiency, Dependability) and hedonic quality (Stimulation, Originality). Pragmatic quality describes aspects of tasks-related quality, while hedonic quality describes aspects of non-task-related quality. Confidence interval for items and scale in this study, 5% confidence interval for the averages of scale and the single item is shown. The confidence interval is a measure for the estimation accuracy of the scale average. The data entered will undergo a process of transformation, calculation of average scores, calculation of variance and standard deviations, and calculation of confidence intervals and comparisons with benchmark data sets [30]. The benchmark data set used was a data set derived from 18483 participants based on 401 research from various application products, such as business applications, web pages, webshops, and social networks. Fig. 4 is the calculation result of UEQ that has been distributed; the graphic image above shows the average score of the User Experience Scale from using the MoLearn application with the Benchmark Data Set. The measurement results showed that the scores of the attributes on the scale of attractiveness, clarity, efficiency, and accuracy are in the category of good scores, while the stimulation and novelty attributes are in excellent scores. The next step is gathering users' written perception in using the MoLearn application. There are several positive perceptions from users of the website and Android-based MoLearn application, namely:  Attractiveness, attractive user interface, and users get comfort in using it; overall, the display is quite good.  Perspicuity, An informative, clear, and easy-to-understand design equipped with a dashboard display is considered complete with important information/ announcements for users, registered school information, teachers, students, latest discussions, new teacher's materials, and new MGMP materials.  Efficiency, Features-grouping based on menu tab that speeds up performance and access to the information needed. The ease for students to do learning, see new material, conduct discussions with other students or with teachers and the available feature for students to be able to do examinations and submit their assignments.  Dependability, a predictable display with notifications for users as well as the use of scheduling features for students  Stimulation, the addition of features such as exam and task submission for the Android-based MoLearn application is very beneficial for students.  Innovative, both application-based website and Android designs are deemed to have fulfilled the creative and innovative elements Under users' perception stated above, the MoLearn Application design has been perceived more positively; this has been in line with the results of data processing obtained from UEQ.

IV. CONCLUSIONS
The results of the study to determine the level of user experience and explore user perceptions of the MoLearn Application by using the User Experience Questionnaire (UEQ) for evaluation showed that 200 respondents who were average high school students gave good marks on attributes of attractiveness, clarity, efficiency and accuracy scales while the attributes of stimulation and novelty are excellent. Students' perceptions of websites and Android-based application design are considered creative, innovative, interesting, easy, and informative attributes.
ACKNOWLEDGMENT This work was supported by Research and Community Services Division, Universitas Dinamika.