Design and Validation of Online Learning Environment Questionnaire

— This study aims at designing and validating the questionnaire as an instrument to assess the lecturers’ perception in terms of factors affecting the ‘ online learning environment’ . Forty participants who have been teaching English completed the 125 items in questionnaire. The result of the study showed that average variance extracted for peer support was 0.66 and instructor support was 0.52 indicating adequate convergence. The construct reliability showed value of 0.83 for peer support and 0.70 for instructor support indicating the good construct validity. Peer support and instructor support constructs in this study were the model fit to the factors affecting the ‘ online learning environment’ .


I. INTRODUCTION
The study related to design and validation of the 'online learning environment' questionnaire in assessing the English lecturers' perception on factors affecting the learning environment are limited. Dunn and Hansford [1] studied an attempt to assist nurse educators in identifying and measuring what factors influence student learning outcomes within the clinical learning environment. The clinical learning environment in the Australian undergraduate nursing student based on its existences and influences was characterized by five factors: relationship between staff and student, commitment of nursing manager, relationships of patient, satisfaction of student, and hierarchy and ritual. Interpersonal relationship has dominant role in clinical learning environment based on students' perception.
Armatas et al. [2] studied that students are required assistance to be familiar with the learning environment. It is difficult for many of students to make well-versed alternatives using the available resources that can discourage the quality of students' learning experience. Hence, teacher's role as studied by Dabbagh as in [3] requires transformation from an expert, knowledge major deliverer, and learning event's organizer to a source, facilitator, instructor, and peer in learning process. Learning as a social process with various and energetic learner population in an online environment necessitate an alteration of teacher's role from instructive to supportive and facilitative. To enable such transformation, it is fundamental for teachers to be accustomed to the use of technology of learning and model of pedagogy.
As the role of teachers transformed to be supportive, teachers should help learners in setting up the strategy of time management. Helping students in deciding the time management strategy may demonstrate to be valuable [4], [5]. Teachers should assist learners by working together to establish community or feelings of networking. Integrating strategies for sense of networking between students and teachers into the course design may support with this endeavour [6]- [8]. This is in a line with the study by Chou and Liu as in [9] who highlighted the thought that the lecturer and students in online environment are required to become reflective practitioners.
Chou and Liu [9] studied that students are more enthusiastic to be present in the class due to the bright means of interacting with peers and instructors through technologymediated virtual learning environments. This system enable the students to ask and answer questions, post comments, and participate in a sharing and exchange of knowledge with peers and the instructors in order students may encompass further possibilities to express their up to date insight.
Pearson and Trinidad [10] studied that educators need learning theories knowledge, best practice models in designing and implementing learning online, and feedback in specifying attempts to match the favourite of students' learning environment. Furthermore, Pearson and Trinidad found that questionnaire on Online Learning Environment Survey (OLES) affords a applied approach to collect feedback simultaneously when e-learning environments is being used worldwide.
Another questionnaire developed by Walker and Fraser as in [11] is Distance Education Learning Environments Survey (DELLES), designed to assess the previous instruments on distance education as an essential learning environment. This instrument was developed by Clayton as in [12] with On-Line Learning Environment Survey (OLLES) found that it is only sufficient to conclude tentatively about the reliability and validity of the scales and individual. However, the most popular instrument in 'online learning environment' is what is developed by Taylor and Maor as in [13] as Constructivist Online Learning Environment Survey (COLLES). COLLES suits to teachers, lecturers, and researchers as well as for those who concern on the online academic role to promote the university teaching reform.

II. METHOD
This study aims at designing and validating the questionnaire which measures the factors affecting the 'online learning environment'. This study involved only a one-time response to the 63 statements on the questionnaire. This study was conducted for designing and validating the questionnaire by assessing the perception of EFL lecturers on factors affecting the 'online learning environment' [14]. There were forty lecturers who have been teaching English in Indonesia for years participated in this study by giving responses to 125 statements with fourteen (14) constructs and sixty three (63) statements. Fraenkel and Wallen [15] described that survey study is proposed to obtain data in an attempt to find out the definite characteristics of a group. All the data from respondents were coded and inputted to SPSS 21 software and then analyzed using AMOS 18 software to measure the validity and reliability. The researchers applied the Confirmatory Factor Analysis (CFA) to find out the convergent validity including factor loading (L i ), Average Variance Extracted (AVE), and Construct Reliability (CR).

A. Questionnaire Design
In a questionnaire design, constructivist learning environment, constructivist classroom, and social learning theory were used as a model to predict the factors affecting the 'online learning environment'. Theory of constructivist learning environment viewed that online environment can engage students in meaningful learning based on individual interests to form social community that can encourage or discourage academic honesty [16].
Furthermore, constructivist classroom theory viewed that teacher's role is facilitator that provides students with various experiences to build learning and maximize social interactions between students to encourage negotiating meaning [17]. Besides, theory of social learning thereby social interaction in 'online learning environment' is essential factor for students in sense of learning community and interaction among peers and instructors [18].
The related previous researches studying the 'online learning environment' were reviewed to locate the questionnaire measuring the factors affecting the 'online learning environment'. Walker and B. J. Fraser [11] developed the Distance Education Learning Environments Survey (DELLES) involving 34 statements with 6 construct. The survey was designed for higher education students. Therefore, the researchers modified the statements which designed for lecturers' perception. Thus, the researchers adapted 33 statements and removed 1 statement to avoid the ambiguity in giving response on questionnaire.
Clayton [12] developed the Online Learning Environment Survey (OLLES) consisting of 7 statements with 7 constructs. The survey was designed to capture the perception of students in 'online learning environment'. Hence, the researchers adapted and developed the questionnaire by adding 1 statement in material environment construct. All the 8 statements were modified to acquire the perception of lecturers in 'online learning environment'.
Taylor and Maor [13] developed the Constructivist On-Line Learning Environment Survey (COLLES) to support the internet use for delivering professional development to postgraduate students and teachers of science, mathematics, and technology. The researchers used "COLLES Example: Actual" to investigate the lecturers' perception on actual 'online learning environment'. The survey consisted of 24 statements by 6 constructs. The researchers adapted 22 statements and removed 2 statements by considering the applicability of the questionnaire.

B. Questionnaire Validation
All 63 statements that were adjusted, 3 statements that were removed, and 1 additional statement were based on the recommendation by three experts who independently reviewed the questionnaire for content and face validation. Response to all statements were offered in a five-point Likert scale where 1 indicated strongly disagree, 2 indicated disagree, 3 indicated neutral, 4 indicated agree, and 5 indicated strongly agree. Approval for this study was obtained from Measurement and Evaluation Program of Education Faculty, Universiti Teknologi Malaysia, Language Center of Universitas Negeri Makassar, and English Department of Universitas Muhammadiyah Makassar, South Sulawesi, Indonesia.

III. RESULTS AND DISCUSSION
The researchers analysed the data based on the results of the study to measure the convergent validity. Hair et al. [19] explained that the items are indicators of a definite construct should share a high amount of variance in general. The researchers provided the detailed value of the convergent validity including factor loadings, average variance extracted, and construct reliability.

A. Factor Loading
The value of loading estimate is a main concern in SEM. High loading value is considered as high convergent validity on the latent construct. All loading estimate value should be statistically significant. Normally, the standardized loading estimates are constrained to range between -1.0 and +1.0, whereas the unstandardized loading correspond to covariance without upper or lower bound [19].
Standardized loading factor should be within 0.5 to 0.7 or higher [19], [20]. This loading estimate or larger confirmed that the items are firmly connected to construct associated and an indication for construct validity. The researchers applied the standardized loading estimates, which remove effects due to the scale of the measures due to the more difficult to interpret unstandardized estimate.
Based on Hair et al. in [19] that construct with fewer than three indicators should be avoided and factor loadings below 0.5 suggest variable is a candidate for deletion from model. Therefore, this study remain two constructs, peer support and instructor due to the case of standardized loading factor. These two factors, in this study, are considered as factors affecting the 'online learning environment'. Table 1 below represents the standardized loading estimates of each item or indicators of peer support and instructor support. The construct of peer support can be defined as encouraging support provided online by fellow students [13]. It can be represented by the four items on peer support (see Table 1).The construct of instructor support can be defined as hopeful support from instructor to provide feedback, time, and help in online learning [11]. It can be represented by the four items on instructor support (see Table 1).
Factors affecting the 'online learning environment' are represented using path diagrams. The path diagram as in Figure 1 below indicates the relationships between constructs of peer support and instructor support. Figure 1 below also shows the relationship between constructs peer support and instructor support and their associated items or indicators. The relationship between the latent constructs (peer support and instructor support) and their items or indicators are represented by arrows from the constructs to the items where each item has an error terms (e).
Model Fit: Chi-square = 43.888, df = 19, CFI = .852, p = .001. Comparative Fit Index (CFI) range from 0 to 1.00 are derived from the comparison of a hypothesized model with the independence (or null) model [21]. CFI value of > .90 is considered as a model fit [22].

B. Average Variance Extracted
The AVE value can calculated using standardized loadings estimate as in a formula: The L i indicates the standardized loading estimate, i is the number of items, and n is items. AVE is computed as the sum of squared standardized factor loadings which can be found in the AMOS 18 output as squared multiple correlations divided by the number of items. Finding out the AVE value of each construct is required for both the peer support and instructor support constructs for convergent validity. The AVE in CFA is calculated as mean of variance extracted for the items loading of a construct and is a summary indicator of convergence [23]. An AVE value of 0.5 or higher indicates adequate convergence [19].

C. Construct Reliability
Reliability in the case of convergent validity is construct reliability. Construct Reliability (CR) is an indicator of convergent validity which is often used in SEM models. CR is the squared total of factor loadings (L i ) of a construct and the sum of the error variance (e i ). Hair et al. [19] suggested that reliability estimate value of 0.7 or higher indicates good reliability. Reliability between 0.6 and 0.7 is acceptable. CR is one of indicators of good construct validity of a model. It is computed from the squared total of all factor loadings (L i ) for each construct and the sum of the error variance for a construct (e i ) as:   (15) 71 . 0 = CR (16) The researchers administered the questionnaire to lecturers where English is taught as Foreign Language (EFL) to find out the actual factors of 'online learning environment' in EFL academic setting. Fourteen (14) constructs with sixty three (63) statements that were adjusted remained 2 constructs and 8 statements due to the standardized loading estimates.
The researchers deleted the items which have standardized loading estimate below 0.5 to meet the high convergent validity on the construct. The significant difference between the standardized loading estimates and unstandardized ones based on the value range of standardized loading factor between -1.0 and +1.0, whereas the unstandardized loadings connected to covariances without higher or lower border [19]. By remaining the standardized loading estimates at a minimum of 0.5, the average variance extracted of peer support of 0.66 and instructor support of 0.52 showed adequate convergence. Whereas, the construct reliability for peer support of 0.83 and instructor support of 0.71 indicated the good reliability. Due to the construct reliability is one of indicators of construct validity, the constructs of peer support and instructor support in this study showed the good construct validity.
The two (2) remaining constructs were not anticipated. The result of this study showed that peer support and instructor support are fundamental for students' engagement to produce meaningful learning in a social community to negotiate meaning, which is expected to encourage academic honesty [16], [17]. In addition, the result of this study showed the importance of interaction among peers and instructors in 'online learning environment' [18].

IV. CONCLUSIONS
This study was conducted to design and validate the questionnaire by assessing the lecturers' perception on factors affecting the 'online learning environment'. Forty EFL lecturers responded to 125 statements. Fourteen (14) constructs with sixty three (63) statements that were adjusted remained 2 constructs and 8 statements due to the standardized loading estimates. The result of this study showed that average variance extracted for peer support and instructor support indicating adequate convergence and the construct reliability for both of the remaining constructs indicating the good construct validity. Constructs of peer support and instructor support in this study showed model fit on factors affecting the 'online learning environment' in EFL academic setting.