Internet Learning Volume 5, Number 1, Fall 2016/Winter 2017 | Page 12

was 4.3 and 3.5, respectively. Online Graduate Course Evaluation from Both Students’ and Peer Instructors’ Perspectives Utilizing Quality Matters TM All of the courses received higher ratings for the key components of QM, including learning objectives, assessment and measurement, instructional materials, course activities and learner interaction, and course technology (Categories 2–6). Courses appeared to require improvement in the following categories: course orientation, learner support, and accessibility (Categories 1, 7, and 8). Inter-rater Reliability between Students and Peer Instructors Overall, students evaluated the courses more positively than the peer instructors did for all the courses except one (D004). However, the differences did not appear to be great (ranged from 0.1 to 0.6). We were more interested in checking whether student evaluations were consistent with the evaluations of peer instructors. In order to see the relationship between the students’ and instructors’ evaluations, researchers carried out Pearson’s correlation analysis. The analysis was conducted separately for each course. To compare the two groups’ evaluations, researchers calculated the average scores of each evaluation item and used them to calculate the Pearson’s correlation coefficients. The analysis revealed that students’ evaluations of courses were significantly correlated with the instructors’ evaluations in all the courses. Those ranged from r = .34 to r = .67 (see Table 2). Reviews of Essential Standards On the basis of the findings, we confirmed that the courses fulfilled the key components of QM standards except in the course technology category. In order to gain deeper understanding of the evaluation, researchers reviewed the 21 essential standards (see Table 3). The findings revealed that the strongest areas of the courses were learning objectives, assessment and measurement, and learner engagement. Regarding the learning objectives, the following standards were highly evaluated: learning objectives or competencies were (1) suited to the level of the courses; (2) measureable and consistent with the course level goals; and (3) clearly stated from the learner’s perspective. According to the results, the courses included learning activities that promoted active engagement and achievement of learning goals. Policies about course grades were stated clearly. Table 2. Correlations of Evaluations between Students and Instructors Note: The number of questions for each course is 43. So, the degrees of freedom is 41. 11