Skylight worked with MDS leadership to develop a program evaluation plan for the new professional masters. It included a mid-program evaluation survey and an end-program evaluation survey. The intent of the mid-program survey was to collect information on student satisfaction with the MDS Program so far as well as student perceptions of their learning and learning experiences in the program. The timing of the survey allowed MDS instructors to make adjustments for the remainder of the program. An end-program evaluation survey is under development.
Motivation
The Master of Data Science Program was offered for the first time in 2016. With any new program, feedback is important for evaluating student satisfaction as well as perceived learning so that adjustments can be made to fine-tune future iterations of the program. Skylight program evaluation initiatives contribute to better student learning and experience in new programs in the Faculty of Science by enabling teaching team make informed decisions.
Outcomes and impact
An iterative process involving Skylight members and all members of the teaching team eventually led to a 50-question online survey that was administered online in January 2017. The questions were mostly adapted from validated survey instruments. Most of the questions were multiple choice (Likert scale) with some having open-ended follow-up questions.
The Master of Data Science teaching team used the survey results (as formative feedback) together with their experience in the first half of the program to make adjustments to the second half of the program. They are also using the data to inform planning for the next iteration of the program, which will begin in September 2017.
Skylight’s contribution
Skylight consulted with the MDS teaching team to better understand the goals of the evaluation, then proposed a comprehensive plan. The team proposed surveys exploring student satisfaction and perceptions of learning at roughly the midpoint of the program and again at the end. In the interim, Skylight suggested short, in-class questionnaires to obtain snapshots of student satisfaction. After researching survey instruments, questions from validated surveys were adapted for the MDS evaluation in consultation with the teaching team. A final report, including statistics and a series of recommendations for the teaching team to support program revision decisions was produced. Skylight is now developing the end-of-program survey (to collect summative feedback). Together with the mid-program survey, the data will enable the MDS teaching team to better under student satis outcomes.