A Novel User Experience Study of Parallax Scrolling using Eye Tracking and User Experience Questionnaire

— Parallax scrolling technique is being devoted as a unique and an innovative trend in the web design. Parallax scrolling provides 3D perception on a web page. Previous works observed user experience issues of parallax scrolling merely based on subjective questionnaires. Their findings leave a research question whether the results are valid, as participants may perceive a written questionnaire differently. Additionally, bias and ambiguity in the questionnaire can affect the research results significantly. To solve this research problem, we present a novel user experience study of parallax scrolling in storytelling and online shop website using eye tracking and User Experience Questionnaire (UEQ). Forty ( N =40) participant joined the experiment on a voluntary basis. Each participant only interacted with one out of two websites (storytelling or online shop) and only one effect (with or without parallax scrolling). We found that parallax scrolling affected UEQ score of Attractiveness of the storytelling website ( p < 0.05). Our findings suggest that parallax scrolling improves user engagement in storytelling website. We also observed that the participants spent time almost two times faster to find an object of interest in an online shop with parallax scrolling compared with the similar task in an online shop without parallax scrolling ( p < 0.05). We thus argue that parallax scrolling is useful during interacting with particular websites that require visual object localization. In future, web designers should consider the appropriate usage of parallax scrolling to optimize user experience while avoiding additional distraction caused by this technique.


I. INTRODUCTION
Unique and innovative web design is critical for attracting the attention of the users [1]. A recent finding suggests that functionalities are not considered as a single aspect of user satisfaction [2]. Usability and efficiency are two other factors that affect user experience on the web [3]. To deal with these matters, the techniques of web design have undergone several changes over the past three decadessuch as the use of static pages, dynamic pages, information aggregation, user-based content generator, responsive design, and parallax scrolling [4].
During the 1990s, static web design was used primarily to provide one-directional information.
During the development of the Web 2.0 in the 2000s, dynamic web design has been used to incorporate user-generated content. Since the launch of the iPhone in 2007, web developers have included responsive design as information is accessed on several platforms including handheld computers, tablets, and smartphones [5]. Responsive design is needed for consistent layout and preserving the structure of web navigation. In the last five years, the parallax scrolling has been well-known among web developer working on interactive media, company profiles, and even online shops [6] - [9].
Parallax scrolling is a technique of web design that provides 3D perception on a web page [6], [7]. The method implements the concept of depth perception by motion parallax. The depth illusion on a flat 2D screen is developed from different scrolling speed between background and foreground elements of the website. In summary, scrolling and depth perception are two critical factors for successful parallax scrolling technique. The use of parallax scrolling on various websites is expected to increase user engagement as well as the attractiveness of the websites. Instead of using multiple pages, a website with parallax scrolling puts all contents in a web page while navigates the visitors back and forth across the contents. The popularity of parallax scrolling was amplified by its use in the New York Times website that presented an article about snowfall at a tunnel creek [10]. Besides news website such as the New York Times, Jaffe also found that parallax scrolling has been increasingly used in websites of commercial product presentation and storytelling [11].
Rapid usage of parallax scrolling in various websites have attracted different research groups to scientifically prove the effectiveness of this technique (see Table 1). In 2013, Yamin and Jaafar investigated the effect of parallax scrolling in a website of a national museum [9]. They compared three web designs: a design with corporate characteristics and parallax banner effects (1 st design), a design with playful characteristics and parallax scrolling (2 nd design), and a design with playful and corporate features but without parallax scrolling (3 rd design). They found that participants reported the most engaging experience during interaction with the 2 nd design.
In 2015, Frederick et al. [8] conducted research to investigate effects of parallax scrolling on perceived usability and user satisfaction. Frederick et al. studied parallax scrolling by measuring subjective report on satisfaction, usability, enjoyment, visual appeal, and fun. They invited 86 participants and randomly divided them into two groups, each of which interacted with a parallax scrolled and a non-parallax scrolled website of a mock-up hotel. In their research, the participants reported that parallax scrolling improved the attractiveness of the website in a hedonic and fun context. In contrast to previous study of Yamin and Jaafar [9], some participants also reported that parallax scrolling induced motion sickness and reduced perceived usability of the website. Additionally, Fu [7] also stated that the use of parallax scrolling in a mobile interface may produce usability issues as users are easily lost in navigation while they face difficulty in accessing the primary navigation bar.
In 2015, Ku conducted further investigation of parallax scrolling [1]. Ku did two studies. In the first study, Ku observed general perception of parallax scrolling in various websites. Ku presented a storytelling website, a commercial product presentation website, a company profile website, and an info graphic website about fruits. In the second study, Ku compared effects of parallax scrolling in parallax scrolled and non-parallax scrolled of info graphics and storytelling website. Ku reported that participants agreed about the use of parallax scrolling in hedonic contexts. However, the use of parallax scrolling is context dependent. In this case, parallax scrolling is more suitable for non-goaloriented instead of goal-oriented context. Hence, parallax scrolling is preferable in a website with hedonic nature (i.e., a non-goal oriented website such as info graphics web) compared with a goal-oriented website (i.e., a storytelling website).

Authors
Year

This research 2018
Investigating In 2017, Wang and Sundar [6] conducted a more recent user experience study of parallax scrolling. They investigated effect of dimension (with vs. without presence) and interaction technique (clicking vs. scrolling) on several dependent variables. The variables were perceived vividness, perceived coolness, natural mapping, and perceived ease of use, user engagement, and attitude toward the website, behavioral intention toward the website, attitude toward the product, and behavioral intention toward the product. They presented four websites of Samsung Gear product, each of which was manipulated based on the studied independent variables. Wang and Sundar argued that parallax scrolling improves both perception of coolness and vividness while positively associated with the reported perceived ease of use and user engagement. However, they also found that if the users said parallax scrolling did not improve perceived vividness, coolness, and ease of use, the users were less likely engaged with the content of the website.
The researches as mentioned earlier used self-reported questionnaires to observe the effects of parallax scrolling on user experience. Their findings still leave a research question whether the results are valid as each participant while bias may perceive a written questionnaire differently and ambiguity in the questionnaire can affect the research results significantly [12]. These drawbacks can be resolved by implementing behavioral or physiological measurements, such as eye tracking and electrocardiography [13][14][15][16][17][18][19]. To the best knowledge of the authors, however, previous works lack sufficient understanding on how parallax scrolling affects visual object localization while accessing information in a website. To fill this research gap, we present a novel user experience study of parallax scrolling in a storytelling website and an online shop using User Experience Questionnaire (UEQ) and eye tracking.

A. Participants
We recruited forty participants (N = 40) on a voluntary basis. The participants were undergraduate and graduate students of the Department of Electrical Engineering and Information Technology, Universitas Gadjah Mada, Indonesia. Twenty-five out of forty participants were males, and the rest were females. About 47.5% participants were 21 years old, 40% participants were 22 years old, 5% participants were 19 and 23 years old, and the rest were 20 years old. All participants were healthy and did not wear prescription glasses or contact lenses. Before the experiment began, all participants were asked to complete informed consent forms. All authors confirmed that the study was arranged according to WMA Declaration of Helsinki (Ethical Principles for Medical Research Involving Human Subjects). The number of participants complied with minimum recommended statistical power for experimental research [20]. Figure 1 shows the experimental setup of this study. We used GP3 eye tracker (Gazepoint Research Inc., Canada) to record and to analyze fixational eye movement. The duration of fixational eye movement is about 200-300 ms. Fixation is  considered as the most critical metric in eye tracking research as it also shows a quantitative amount of attention at a particular time.

B. Apparatus
The eye tracker was mounted beneath the secondary monitor (see Fig. 1). According to the manufacturer, the eye tracker implemented the Pupil-Corneal Reflection (P-CR) method to measure the direction of the gaze. The eye tracker is equipped with near-infrared (NIR) camera, infrared illumination sources, and visible light filter to produce high contrast video regardless light intensities of experiment room The sampling rate was 60 Hz, with 0.5−1 degree accuracy, and less than 0.1 degree of spatial resolution. The operation range was 50-80 cm from the computer screen with less than 50 ms of system latency, providing sufficient support for this study. Recording and analysis of eye movements data were performed in Gazepoint Analysis UX Edition v2.12.1 (see Fig. 2).
To record the eye movements data, we used a Toshiba Satellite L745 personal notebook with Intel Core i3 M380 2.53 GHz CPU, 2 GB RAM, DirectX 10, Intel HD graphics card, 1366 x 768 pixels resolution, and Windows 10 operating system (32-bit). To present the stimulus, we used Samsung S19C150 18.5 inch monitor with 1366x768 pixels resolution. The eye tracker was installed beneath the secondary monitor (see Fig. 1). The secondary monitor was positioned about 60 cm from the participant.

C. User Experience Questionnaire
User Experience Questionnaire (UEQ) has been widely used to observe user experience toward a specific system, such as interactive products, websites, web services, and social networks [21]. The goal of UEQ is to allow fast and comprehensive assessment of user experience. The UEQ consists of 26 questions with seven points score (e.g., 1-7). The questions can be classified into six scales: (i) Attractiveness, (ii) Perspicuity, (iii) Efficiency, (iv) Dependability, (v) Stimulation, and (vi) Novelty.
Attractiveness is related to user's overall impression on the observed product. Questions related to attractiveness include several keywords, such as annoying or enjoyable, bad or good, unlike or pleasing, unpleasant or pleasant, unattractive or attractive, and unfriendly or friendly. Attractiveness represents a pure acceptance or rejection of the product without compelling argument on users' impression.
Perspicuity, Efficiency, and Dependability can be associated with goal-directed (pragmatic) aspect of the system. Perspicuity reports whether the product is easy to be learned while the user is getting familiar with all features in the product. Efficiency is related to the ability of the product to help the users getting their tasks done without unnecessary effort. Dependability measures whether the users can control the flow of the system during the interaction. For example, the users may report whether the system is predictable or unpredictable, obstructive or supportive, and not secure or secure.
Stimulation and Novelty can be associated with nongoal-directed aspect (hedonic) of the system. Stimulation is used to observe whether the users are excited and motivated to use the product. This scale may include some keywords such as boring/exiting and not interesting/interesting. On the other hand, Novelty measures whether the users perceive that the product is creative or innovative.
The neutral UEQ scores lay between -0.8 and 0.8. The system will be perceived as bringing good user experience if the average UEQ scores is more than 0.8. On the contrary, the system is said to negative user experience if the average UEQ scores is less than 0.8. An extremely good system will have a UEQ scores between 1.5 and 2 [24]. Figure 3 and 4 show the configuration of four stimuli used in this study. As recommended by Jaffe [11], we designed two types of website for this study-storytelling and online shop website. We presented two storytelling websites, each of which was designed with parallax scrolling (coded: story-p) and without parallax scrolling (coded: storynp). The content of both storytelling websites was in Bahasa Indonesia, displaying information about Batik-traditional hand-dyed cottons and silk garments that were inscribed by the United Nations Educational, Scientific, and Cultural Organization (UNESCO) in 2009 as an intangible cultural heritage of humanity [22]. On the other hand, the online shop websites presented various types of casual and sports shoes. The online shops were designed with parallax scrolling (coded: shop-p) and without parallax scrolling (coded: shop-np). The online version of all websites with accessible navigation can be found in the following URL: http://wibirama.staff.ugm.ac.id/parallaxweb

D. Design and procedures of the experiment
We implemented a between-group design. The design aimed to avoid learning effect during the experiment. For the storytelling websites, we asked the participants to find information about Batik as a world heritage as stated by the UNESCO. For the online shop websites, we asked the participants to look for and to read detailed information about "Adidas Yeezy Boost" shoes by hovering the mouse pointer over the product. Thus, there were four different experimental groups for this study (i.e.,: story-p group, story-np group, shop-p group, and shop-np group). Forty participants were randomly distributed into four groups so that each participant only interacted with one interface and completed one particular task.    5 Step-by-step procedures of the experiment. Figure 5 shows step-by-step procedures of the experiment. Before the experiment started, the participant was asked to fill in a consent form. A brief explanation of experimental procedures was delivered to the participant. Next, the participant was asked to sit in front of the secondary monitor. The participant's gaze was then calibrated with standard nine points calibration. After the calibration finished, the participant was asked to complete the task not more than three minutes. After the task was completed, the participant was then asked to fill in the User Experience Questionnaire (UEQ). The duration of the experiment for each participant was ten minutes.

E. Validation of the experimental results
Before extracting quantitative values of fixational eye movements, Area of Interest (AOI) on the presented stimuli was defined (see Figs. 6 and 7). In the storytelling websites, the AOI was a paragraph containing a brief explanation that the United Nations Educational, Scientific, and Cultural Organization (UNESCO) have inscribed Batik in 2009 as an intangible cultural heritage of humanity. In the online shop websites, the AOI was a picture and information about "Adidas Yeezy Boost" shoes. From the defined AOIs, we extracted the duration of dwell time before the first fixation at the AOI. This metric shows the cognitive load needed by the participant to find the AOI (measured in seconds) [23].
The UEQ data were grouped and analyzed according to experimental groups. The collected UEQ scores were transformed by subtracting 4 from every score. The transformed data were plotted between -3 (bad) and 3 (excellent). Furthermore, the UEQ scores were averaged.
We conducted statistical analysis for the eye tracking data and UEQ scores. We performed two-way ANOVA betweengroups design with two independent variables: types of web (online shop web vs. storytelling web) and use of parallax effect (with and without). The dependent variable was time to first fixation.
To observe the effect of parallax scrolling on each type of website, we conducted Independent Samples T-test. The test was performed separately for online shop and storytelling web. The independent variable was the use of parallax effect. The dependent variable was the time to first fixation. We also conducted Mann-Whitney U test to compare averaged scores of the UEQ scales: (i) Attractiveness, (ii) Perspicuity, (iii) Efficiency, (iv) Dependability, (v) Stimulation, and (vi) Novelty. The independent variable was the use of parallax effect (with vs. without). The dependent variable was UEQ score for each scale. The test was performed to observe whether there was a significant effect of parallax scrolling on the UEQ scores. The test was conducted for all websites.        Figure 8 shows the results of eye tracking data for each group of the experiment. We analyzed time to first fixation (TTF) on the designated Area of Interest (AOI). From the descriptive experimental results, we observed that parallax scrolling improved user engagement on both storytelling and online shop website. The results show that participants using a web with parallax scrolling spent time faster than their counterparts to find the designated AOI. Based on the results of two-way ANOVA, we did not find significant effect of the types of website and the use of parallax scrolling toward TTF scores. However, results of Independent Samples T-test show that there was significant difference in the TTF scores for an online shop with parallax scrolling (M=23.53, SD=11.60) and an online shop without parallax scrolling (M=13.48, SD=4.54); t(18)=-2.551, p < 0.05). Additionally, the effect of parallax scrolling on the TTF data was not significant in the storytelling website. The descriptive results, however, show that the average TTF score for storytelling web with parallax scrolling is lower than the average TTF for storytelling web without parallax scrolling. Figure 9 and 10 show the UEQ scores of storytelling web without and with parallax scrolling, respectively. The average UEQ scores for storytelling website without parallax scrolling are 1.083 (Attractiveness), 1.025 (Perspicuity), 1.375 (Efficiency), 1.125 (Dependability), 0.875 (Stimulation), and 0.575 (Novelty) (see Fig. 9). The average UEQ scores for storytelling website with parallax scrolling are 1.600 (Attractiveness), 1.600 (Perspicuity), 1.025 (Efficiency), 1.150 (Dependability), 1.175 (Stimulation), and 1.125 (Novelty) (see Fig. 10). Figure 9 and 10 denote that parallax scrolling slightly improved all UEQ scales except Efficiency. We observed that the score of Efficiency slightly decreased from 1.375 (without parallax scrolling) to 1.025 (with parallax scrolling), despite the later was still above the average standard of positive user experience. We also observed that the participants reported UEQ scores of Stimulation and Novelty for the storytelling website without parallax scrolling. In particular, the Novelty score of storytelling website without parallax scrolling was below the standard average of positive user experience (0.575). On the other hand, the UEQ scores for both scales were improved with the addition of parallax scrolling in the storytelling website. Using Mann-Whitney U test, we found a significant effect of the use of parallax scrolling on UEQ scores of Attractiveness with p-value of 0.019 (U=19.5, W=74.5, p < 0.05). We did not find any significant effect of parallax scrolling on the other UEQ scales. Figure 11 and 12 show the UEQ scores of online shop websites without and with parallax scrolling. The average UEQ scores for online shop website without parallax scrolling are 1.600 (Attractiveness), 1.325 (Perspicuity), 1.150 (Efficiency), 1.350 (Dependability), 1.750 (Stimulation), and 1.075 (Novelty) (see Fig. 11). The average UEQ scores for online shop website with parallax scrolling are 1.867 (Attractiveness), 1.700 (Perspicuity), 1.800 (Efficiency), 1.375 (Dependability), 1.650 (Stimulation), and 1.650 (Novelty) (see Fig. 12).

III. RESULTS AND DISCUSSION
As depicted in Fig. 11 and 12, we observed that score of Stimulation slightly decreased from 1.750 (online shop without parallax scrolling) to 1.650 (online shop with parallax scrolling). Despite of this, the score fell above 0.8, hence categorized as Good. The parallax scrolling also improved UEQ score of Efficiency. In this case, we observed that parallax scrolling encouraged the participants to finish the task with unnecessary effort. Based on the Mann-Whitney U test, we found a moderate effect of parallax scrolling on Efficiency (U=29, W=84; p = 0.109). There was no significant difference of the other UEQ scales in online shop with and without parallax scrolling.
Based on the eye tracking data, we learned that parallax scrolling induces positive user experience, particularly for a task that includes localization of visual object. In the case of the online shop, the participants were asked to find information about "Adidas Yeezy Boost" shoes by visually scanning at several pictures in the online shop. This task is predominantly related with the ability of the participants to find suitable object of interest by looking at a picture instead of reading a text. Our results confirm previous research by Ku [1]. In his research, Ku argued that parallax scrolling is more suitable to improve the efficiency of a website with hedonic nature (e.g., an info graphic web) instead of a goal oriented website (e.g., a storytelling web). In this case, we observed similar trend when the participants accessing an online shop web with parallax scrolling. Our results also confirm previous work of Wang and Sundar [6]. They found that parallax scrolling was positively associated with ease of use. In our case, the participants spent less cognitive effort to find the designated AOI in an online shop with parallax scrolling.
The UEQ data show that parallax scrolling improved several UEQ scales. In both storytelling and online shop websites, parallax scrolling improved scores of Attractiveness. A significant effect of parallax scrolling on Attractiveness was found in storytelling website (p < 0.05). This result confirms previous works, for instance, findings of Frederick et al. [8], Yamin and Jafar [9], as well as Wang and Sundar [6]. In this case, parallax scrolling is an important aspect to improve overall impression of the website as Attractiveness reflects pure valence dimension that includes pragmatic quality and hedonic quality of the website [21].
However, the use of parallax scrolling should be accompanied with careful consideration. In our case, we found that parallax scrolling slightly reduced Efficiency in the storytelling websites and Stimulation in the online shop websites. Parallax scrolling might be perceived impractical while the participants were looking for an AOI of text stating that "Batik has been inscribed by the United Nations Educational, Scientific, and Cultural Organization (UNESCO) in 2009".
This was not the case in the online shop websites. Surprisingly, the score of Efficiency was improved by adding parallax scrolling in the online shop website. This result was confirmed by faster time to first fixation (TTF) score from the eye tracking data. Despite this, score of Stimulation slightly lower with the addition of parallax scrolling, showing that the participants might perceive the scrolling was not interesting. Thus, we argue that parallax scrolling is useful to improve efficiency during interacting with particular websites that require visual object localization, such as finding information in an online shop [1]. While parallax-scrolling might not so useful to improve Stimulation of an online shop, this technique is appropriate for improving Attractiveness of a website that is goaloriented and predominantly filled with text instead of pictures. Further investigation using physiological measurement-such as electrocardiography (ECG)-is useful to objectively confirm such emotional arousal on the use of parallax scrolling.

IV. CONCLUSION
Previous studies on parallax scrolling mainly implemented self-reported questionnaires to observe effects of parallax scrolling on user experience. However, their findings still leave a research question whether the results are valid as each participants while bias may perceive a written questionnaire differently and ambiguity in the questionnaire can affect the research results significantly. To fill this gap, we present a novel user experience study of parallax scrolling using eye tracking and user experience questionnaire (UEQ). We aim to resolve the drawback of solely using questionnaire by implementing behavioral measurements-hence, confirming subjective judgment with objective measurement. We presented four stimuli to forty (N=40) participants. The participants were divided into four experimental groups. Each group interacted with only one stimulus-one out of two websites (i.e., online shop or storytelling) with or without parallax scrolling. We analyzed time to first of fixation of eye tracking data and UEQ scores. We observed that parallax scrolling improved time to find visual object of interest in the online shop (p < 0.05). We also found that parallax scrolling affected Attractiveness of the storytelling website (p < 0.05). We thus argue that parallax scrolling is useful to improve efficiency during interacting with particular websites that require visual object localization. Additionally, our findings demonstrate that parallax scrolling improves user engagement on a website that is goal-oriented and predominantly filled with text instead of pictures. In future, our research may be used as a guideline to improve user experience of goal-directed and non-goal directed website.