Alternate Exam Resources

Version information (updates will be made as new information becomes available):

  • Apr 17: New suggestion to add communication channels during the exam (open Collaborate Ultra session along with email). Also incorporated further updates from the Centre for Accessibility in Section 4.
  • Apr 8: Updated links for invigilation (Zoom, Proctorio) to match those at Skylight's main Academic Continuity site and Canvas Quiz/Assignment FAQs in the exam design section.
  • Apr 6: Added links in the exam design section.
  • Apr 3: Major update, breaking content over pages with added detail and recommendations in the integrity section.
  • March 20: Small addition to the integrity pledge.
  • March 19: First version released.

The information in this guide offers ideas and processes for replacing a traditional, in-person, paper-based exam with an alternative that students can undertake remotely.

You may wish to consider the relative grade value in the course of whatever activity is assigned to students for the final exam period (this has been communicated via department channels and isn’t covered in any detail here).

You will need to consider student access to specific electronic equipment. Some students will have limited internet access, some will not have webcams, and/or some will not have access to printers or cameras. If you are thinking of asking students to do things that require a webcam, printer, scanner, and/or camera, communicate with them to determine what’s possible and make alternate plans to accommodate.

With those in mind, the primary recommendations fall into two categories, which have their own sections on this page (you can use the floating right navigation to skip to them):

  • Scheduled exam online for large courses where a take-home test will be too burdensome to grade/process
  • Take-home exams (submitted online) for courses that are capable of offering them, including most smaller courses

Ideas and recommendations (including points adapted from Purdue’s “Teaching Remotely” site):

  • Discover student access needs for your proposed technology: internet and equipment availability will vary among students; prior to the exam period, you can survey students and/or test the technology solution you are intending to use to determine its feasibility and any accommodations that might be needed.
  • In general, for large courses (several hundred students or more), consult with LT support to determine any concerns/adjustments that can help keep the submission process smooth and avoid overloading the servers (for example, staggering the start time or providing students with extra time to account for technical difficulties).
  • Be aware of time zones: students may be in a distant time zone for the exam period, and so extending the response period for the take-home activity could make this more equitable (for example, allowing 24 hours instead of 8-12 hours).
  • Academic Integrity:ideally, you create an assignment that discourages cheating, but in any case, you may also consider asking students to (digitally) sign an explicit integrity pledge and/or a form of online invigilation if necessary (see the "Academic Integrity" section for details).
  • Allowing exams to be open-book/source: assume students will use resources while taking an exam, and even encourage them to do so. Try to ask questions that probe deeper levels of knowledge and understanding, enabling students to apply, assess, and evaluate concepts and facts in meaningful ways. Encourage students to share and cite where they get information from and what resources they use, and be clear about which resources are not to be used, for example include instructions to clarify that any sort of reading might be okay but students should not actively solicit answers from websites.
  • Encourage students to collaborate/share questions and ideas: students will likely work together when they are stuck or confused. You can encourage working in small teams and ask them to include who they work with and in what ways.
  • Focus on solving problems while showing work and explanations: in many cases, students may get the same answer, but showing their work reveals meaningful differences in understanding. Sometimes there may only be a few ways to show work, so you may ask for brief prose explanations or have students record a video of them talking through the process to solve a question.
  • Use question pools: if you have short-answer or multiple-choice questions, create pools in your LMS so that students receive different sets of questions (this can also be done with essays and more complex questions).
  • Use student-generated questions with explanations: instead of trying to ensure everyone answers your limited number of questions on their own, ask every student to create their own question with an explanation of how it would assess a certain topic or skill in a meaningful way. You can also assign students to answer each other’s questions and state whether those questions actually do assess these skills in appropriate ways.
  • Ensure clarity in questions and prompts: especially if your test is timed, your students may not have a chance to ask a question and get a response. It is vital that questions and prompts are clear to novices so your assessment measures what you want it to. Even if not timed, you do not want to be spending your limited time answering clarifying questions.
  • Consider question formats leading to essays, videos, pictures, and other personal responses (on specific course topics or reflection on learning in the course): if your class lends itself to it, having students express their learning through essays, videos, pictures, or other personalized forms of writing/speaking/communicating means that everyone needs to create their own. You can also have students post their responses for each other and assess each other’s work through peer grading (note: some prior training/practice in peer grading for little/no grades is important to making this effective). Rubrics can help guide students as they develop such work and give each other feedback, and, of course, allow your teaching assistants and you a consistent method of assessment.
  • Respect your own time: most of these ideas take time to grade. Try to determine what is feasible in your situation, and use feedback-based or hand-grading intensive assessments sparingly. Also, consider how much feedback students actually need/will use. Many times feedback can be created for the whole group based on common challenges or problems, as opposed to individual responses.

 

1. Scheduled Exam Online
The key recommendation for all of the methods described below is to have a trial run of your exam, ideally with some sort of low/no-stakes real example test for students in the course to try. This will help them understand the process for the exam and will help you to discover any shortcomings or things to fix for the actual exam. More detail on this below. In general, for large courses (several hundred students or more), consult with LT support to determine any concerns/adjustments that can help keep the process smooth and avoid overloading the servers (for example, staggering the start time or providing students with extra time to account for technical difficulties).

1.1 Recommendations for all online testing

  • Highly recommended: provide students with an ungraded test of the process before the exam period if at all possible, perhaps with a couple of questions presented using the intended format. This will help them understand the format and help everyone discover technical issues prior to the actual exam.
  • In addition to this process test with students, review/test your actual test questions in the system with a form of student view to ensure images are visible, etc., before the full test day.
  • Use the already-scheduled exam time (with appropriate buffer) to schedule your online exam; a buffer around the fixed time to account for connection issues. Note that standard exam slots are 3.5 hours to allow for room setup/cleanup, so you can offer a 2-2.5 hour exam with extra buffer time, perhaps 4 hours. This will provide some leeway to students in other time zones and help avoid overlap with students’ other exams as the schedule has already been built based on course registration.
  • Ensure clarity in questions and prompts: especially if your test is timed, your students may not have a chance to ask a question and get a response. It is vital that questions and prompts are clear to novices so your assessment measures what you want it to. Even if not timed, you do not want to be spending your limited time answering clarifying questions.
  • Offer some guides/documentation to help avoid technical difficulties. This can include providing students with guides on the technology and contact info (for you and/or LT support and/or the vendor in some cases) in case something goes wrong. Decide if you want to give them an extra attempt or provide them with more time to account for time lost due to technical issues, or provide enough of a buffer for the submission window relative to the test length to allow for these issues. You also will want a plan for when problems arise (next item).
  • Have a plan in place for students with technical difficulties or other questions. Who should they contact and how can they ask for help? Some instructors have asked students to communicate with them through email, others have created a Collaborate Ultra session and were in the meeting during the exam so students could join the session when they have questions. (Instructor feedback showed that these options worked well and students appreciated knowing that the instructor was available if needed. It gave them a similar experience as a face-to-face exam.) Decide if you want to give them an extra attempt or provide them with more time to account for time lost due to technical issues, or provide enough of a buffer for the submission window relative to the test length to allow for these issues.
  • For a comparison of available technologies and their features, see our Online Exams guide.
  • For exams with handwritten responses, see the section on "Exams with Handwritten Responses".

1.2 Canvas

Canvas quizzes have a variety of question types, and grading can happen directly in Canvas.

  • You can include an “upload a file” question type or comPAIR-style questions to probe higher-order/critical thinking about provided solutions without necessarily requiring students to produce those solutions as part of the test.
  • Note about timed quizzes in Canvas: some users have experienced issues with this, so we would suggest using a quiz with a short submission window rather than a timed quiz.
  • Further recommendation: for larger courses (several hundred students or more), consult with LT support for performance recommendations, like staggering the release time slightly at the start of the quiz.
See our Canvas guide for more information and the UBC Keep Teaching page on Assignments and Assessments for step-by-step guides to setting these up. For a comparison of available technologies and their features, see our Online Exams guide.

1.3 Crowdmark

Crowdmark allows students to submit directly to their system via their "Assigned Assessment" format: questions can be hosted on the Crowdmark platform and students respond with a photo/scan/PDF upload of their answer (they cannot enter answers directly on a web form like a Canvas Quiz). Grading happens online and can be distributed among a team (instructors and TAs).

For a comparison of available technologies and their features, see our Online Exams guide.

1.4 Gradescope

Gradescope supports both on-line “Homework/Problem Set” submission where students upload photos/scans of written answers and “On-line Assignments” (i.e., timed, no-paper needed; can mix multiple-choice questions with open response typed-text questions), as well as uploaded paper-based assessments. Grading happens online based on a rubric and can be distributed among a team (instructors and TAs).

  • Gradescope’s services are available free through June 2020, and so they can be used for free during our April exam period.
  • For more information, see the LT Hub guide for Gradescope.
For a comparison of available technologies and their features, see our Online Exams guide.

1.5 WeBWorK - primarily for Math and Stats courses

Courses that already use WeBWorK may want to use it for all or part of their exam, for example a handful of Math and Stats courses ran online midterms consisting of a combination of a WeBWorK portion and a handwritten portion submitted via Canvas (see "Exams with Handwritten Responses" on this page). Note that WeBWorK has a timed test feature as an alternative to its typical assignments used for homework. On tests, students are not given feedback along the way with each question like they are on homework. While you can learn more on the LT WeBWorK guide page and main WeBWorK resources page, it would be best to consult with people in your department that are familiar with setting up WeBWorK for tests.

1.6 Oral examinations via web conferencing

Again dependent on class size and technology feasibility for students, but you could have each student respond to a few questions live in order to assess their grasp of the course material (this has been piloted in in-person lab courses already). This is time-intensive to do but the subsequent grading time is minimal, and it solves the invigilation issue. You may prefer Blackboard Collaborate Ultra as it allows management of multiple parallel (separate) sessions, so exams could run at the same time with a team of examiners.

1.7 Group/two-stage exams

If you are interested in a group portion for an exam, you could mount an online quiz in Canvas but also provide a Blackboard Collaborate Ultra session (or run multiple sessions concurrently within the same course, if you want to split students into groups), where you split the students in the session into breakout groups (each has its own text chat and whiteboard in addition to the discussion; TAs can circulate from group to group as a way of monitoring things) so that they can complete the group portion of the exam. Unlike in-person group exams where there is usually one response sheet per group, they would each have their own set of responses but would still benefit from the group interaction. This would definitely be something to test prior to the actual exam. Notes and suggestions:

  • Students will need a better internet connection and camera+mic for the group discussion than they will for answering questions or uploading files.
  • Poll your students to ask how they feel about having a group collaboration portion ahead of time.
  • Provide alternatives to the group component where 1) the individual portion is the total grade or 2) take the average of group exam score for students who don't participate in the group portion.
2. Take-Home Exams

Recommended for any course that can manage the grading load, a take-home test would mean that students have a substantial period of time (perhaps 1-2 days) to complete the test and upload their responses.

Response submission possibilities include:

  • If written responses can be submitted in a text format, students could submit via a Canvas assignment by pasting directly into an open response question (or upload a document - see https://community.canvaslms.com/docs/DOC-10151-415241295 for options).
  • For tests that require handwritten responses (e.g., include diagrams or computations that won’t fit in online text boxes), students could be asked to print a test (this may be difficult to achieve consistently with all students!) or work on blank paper and submit completed work digitally, like taking pictures of the pages with their phone and uploading those as pictures or as a PDF; see the section "Exams with Handwritten Responses" on this page for details.
  • More general digital file submissions are also possible: students could upload a document, video, etc.
  • Other platforms may support certain kinds of submissions; for example, students could submit a blog post to a course Wordpress site.
3. Academic Integrity

Key considerations for exam integrity: is the system acceptable to students and to faculty in terms of preserving the integrity of the process, and is the technology feasible? All of the online options are a compromise and offer different approximations of invigilated, in-person exams. This is one reason to reduce the stakes of the exams.

For scheduled exams, a key decision around online testing is whether invigilation will be needed and what system might feasibly support it. Options are quite limited and described below.

3.1 Integrity Pledge

Consider your goals for the end-of-course assessment and if online invigilation will not be feasible, or in any case as students will be completing their exam/assignment remotely, you may consider including an explicit integrity pledge as part of your course. Some tips for integrity pledge use:

Example integrity pledge language

Integrity Pledge (this is currently from Science. Please use until such time as a UBC-wide version is available)

[This top part with the links would be implemented at the course level, and students would need to agree to it once per course.]

I hereby pledge that I have read and will abide by the rules, regulations, and expectations set set out in the Academic Calendar, with particular attention paid to:

  1. The Student Declaration (http://www.calendar.ubc.ca/vancouver/index.cfm?tree=3,285,0,0)
  2. The Academic Honesty and Standards (http://www.calendar.ubc.ca/vancouver/index.cfm?tree=3,286,0,0)
  3. The Student Conduct During Examinations (http://www.calendar.ubc.ca/vancouver/index.cfm?tree=3,41,90,0)
  4. And any special rules for conduct as set out by the examiner.

[This next statement should go at the start of any online assessment in which the integrity pledge is needed/desired.] 

I affirm that I will not give or receive any unauthorized help on this examination, that all work will be my own, and that I will abide by any special rules for conduct set out by the examiner.

3.2 Live Invigilation

Blackboard Collaborate Ultra and Zoom both support large group web meetings with breakout groups, so could be used for live invigilation via webcam. The process would be approximately:

  • Offer enough sessions to cover the whole course where each session fits within the participant limit.
  • In each session, break students into groups, ideally where they can be seen all on one screen by a proctor (e.g., 19 students plus one space for an invigilator).
  • Invigilators can remain within rooms or move between them systematically, depending on personnel. If invigilators keep their webcams off, they could monitor multiple rooms at a time (using multiple browser tabs or devices) but the rooms would not know which was being watched at any given time.
  • You will need clear protocols for how to handle observed misconduct (what evidence should be collected, who to contact, what they will do in the moment or later to contact the student) and student bathroom breaks; these need to be clear to your instructional team and as possible to the students.

For larger classes, Zoom appears to facilitate management of this type of process a bit more than Blackboard Collaborate Ultra, however an advantage of Blackboard Collaborate Ultra is that student names would be tied to their video (they log in via Canvas), making ID checking much more straightforward. Access records would also be available because the system would run through Canvas (which tracks logins, access to materials, etc.), however if Zoom is used and the Zoom link is provided on Canvas, then access records will be generated.

3.3 Computer-assisted monitoring (Proctorio and LockDown Browser)

UBC has a contract with Proctorio, software to monitor students remotely while they are writing a test. If you are planning to use Proctorio for the first time, be sure to gather feasibility information from your students and make alternate plans as needed. Some key information:

  • Proctorio offers different "intensities", which can include video monitoring, audio monitoring, keystroke logging, and preventing/monitoring changing windows away from the browser. To address technology issues like students who are missing a webcam, it is possible to have multiple versions of exams with different Proctorio settings intensities. With strict instructor controls on which students (without webcams, or other access issue) can access a Proctorio exam without the webcam requirement. (from: http://cis.apsc.ubc.ca/proctorio/ )
  • For video and audio monitoring, requires students to have a webcam and microphone enabled while they are writing the test, which provides a recording to the instructional team. Proctorio may create challenges for students with accessibility accommodations; the strict requirements of this program make it incompatible to proctor many students with disabilities. Students with physical disabilities, chronic health conditions (frequent access to washrooms/supplies) or rituals/routines/coping strategies to manage their disability will be unable to set up their writing space to meet the requirements of the software. Students will also be flagged (i.e., for recorded behaviour that the AI thinks requires review) consistently and unnecessarily with this software.
  • Is better suited for online tests (i.e., the student is interacting with their screen) rather than handwritten tests, as the system monitors attention to the screen via eye-tracking.
  • Recordings have to be reviewed systematically (e.g., by looking for anomalies between term-time grades and the exam grade to flag suspiciously high scores), though there are flags for certain kinds of behaviour.
  • Students in countries/jurisdictions with significant firewalls may be blocked from using Proctorio.
  • Proctorio is available for free for the 2019W Term 2 exam period to UBC departments (beyond May is under consideration).

To learn more, see:

Respondus' LockDown Browser software, which students install and prevents them from navigating to other resources during a test, has also been tried with online tests at UBC, but severe limitations have been discovered when using this at a large scale, so we cannot recommend it for large classes. It would not prevent students from using a second device to access other resources (its main applications have been in-person, invigilated online exams to prevent switching between windows, not as much for remote testing). (Guide is here: https://lthub.ubc.ca/guides/lockdown-browser/)

3.4 Exam Question Design

See: Skylight's Canvas Quiz and Canvas Assignment FAQs for common issues discovered in the midterms and exam prep.

For Canvas Quizzes: information about question groups, shuffling answers, time limits (and how to extend for individual students if needed), hiding quiz scores until later, and keeping quiz images hidden before the quiz, see the Canvas page for Quiz Settings to Maximize Security

Ideas for hand-written problems, including an example for converting handwritten responses (where problem solution has several steps) into multiple-choice questions plus short text answers (from Applied Science's Centre for Instructional Support).

Regarding New Quizzes versus Old Quizzes, the current state is that the New Quizzes have some new features but they do not yet cover all the features of the Old Quizzes; see this overview of New vs Old Quizzes from IU, or look over your features of interest in the full feature comparison on the Canvas site.

3.5 Checking student IDs
Students could hold up ID cards to their webcam if being monitored via Zoom. If the students are in Collaborate or Proctorio, their login is attached to their name (a comparison of IDs with photo records could also be done, though).
4. Considerations for Accessibility/Accommodation

Some suggestions to accommodate students, provided by the Centre for Accessibility, are below. Students who have questions about their accommodations should be directed to their CfA advisor. They can also contact info.accessibility@ubc.ca or call 604-822-5844.

For instructors, one-on-one assistance for exam accommodation set up is available from the Centre for Accessibility Exam Coordinators - contact exam.coordinator@ubc.ca.

  1. Canvas is not compatible with the adaptive technologies required by many students with disabilities. Instructors with students requiring this accommodation will be contacted by the Centre for Accessibility. Students will write a computer based exam with live invigilation provided by the Centre for Accessibility using Zoom. Instructors will be required to provide a readable PDF or word version of the exam. Details regarding the process for invigilation will be provided to instructors.
  2. Proctorio is not compatible with the adaptive technologies required by many students with disabilities. Instructors with students requiring this accommodation will be contacted by the Centre for Accessibility. Students will write a computer based exam with live invigilation provided by the Centre for Accessibility using Zoom. Instructors will be required to provide a readable PDF or word version of the exam. Details regarding the process for invigilation will be provided to instructors. Proctorio is also incompatible with many accommodations required by students with disabilities. For example, students with chronic health conditions may require frequent access to a washroom or the ability to move positions during exams. In other cases, students may have rituals or routines associated with their disability or their coping strategies and will be extremely anxious to have their instructor or TA invigilate their exam. The CfA may invigilate in some cases, alternative platforms may be deployed for this purpose, and academic concession may be the result if no suitable solution exists; you can email info.accessibility@ubc.ca with questions if they have not been in touch already.
  3. Extra time accommodation: please manage this for each student based on their specific accommodation. Students should have provided you with their accommodation letter outlining their time extension needs, otherwise you may also log in to our online exam system to verify each student’s extended time requirements. If you require assistance with this process please contact us at exam.coordinator@ubc.ca. Note: extended time accommodations do not apply in the case of take-home exams where students have multiple days to complete an exam; students who believe they need further time allowances should connect with their advisor as they do under normal circumstances.
  4. Adjusted start time: Some students will require the opportunity to write exams at specific times of day due to the impact of their disability or on-going medical condition. Instructors may be required to allow access to an exam earlier than the scheduled start time. Students with this accommodation will be asked to sign a confidentiality agreement. For additional information, contact exam.coordinator@ubc.ca.
  5. Alternate format: the ideal format for a soft-copy exam is a Word (.doc) file that is straightforward and does not utilize significant formatting (graphs and tables etc.) If you need assistance with this or your student requires other formats such as audio please contact exam.coordinator@ubc.ca.
  6. Private or distraction-reduced environment: students will be responsible for finding a suitable space to write their exam.
  7. Students requiring in person supports: Scribe, for example. There are very few cases requiring this and we will work them out on a case by case basis. Please ask the student to contact their advisor.
  8. Spellcheck: Spellcheck will not function fully in Canvas on Lockdown or in Proctorio. Students eligible for this accommodation will only see red underlines. Please allow for spelling errors without penalty if you require students to use the Lockdown function or Proctorio.
  9. Adaptive software/technology: most students who require this technology should already have it on their personal computers. Students may require an alternate format of your exam to utilize their software outside of Canvas, such as Read and Write Gold.

An additional accessibility concern with online exams and international students returning to home countries is that their scheduled exam time may be during the night in their time zone. This is something to consider with take-home exams, perhaps extending the notion of a “full day” for responses to be completed.

For further background in terms of principles for accommodations, UBC’s Centre for Accessibility page on Exam Accommodations (for typical exam periods) is: https://students.ubc.ca/enrolment/academic-learning-resources/academic-accommodations-disabilities/exam-accommodations-guidelines-expectations.

5. Exams with Handwritten Responses

Handwritten response submission possibilities include:

Grading via Online Platforms

You can distribute the grading load among an instructional team (including TAs) by grading online.

Canvas

You can use rubrics and have multiple graders using SpeedGrader: https://community.canvaslms.com/docs/DOC-12774-415255021.

Warning: do not ask multiple graders to enter grades at the same time in SpeedGrader or they will overwrite each other’s work; download all files and have one person enter grades.

Crowdmark

Crowdmark allows students to submit directly to their system through the use of “Assigned Assessments”: questions can be hosted on the Crowdmark platform and students can respond with a photo/scan/PDF upload of their answer (they cannot enter answers directly on a web form like a Canvas Quiz) or a PDF generated from a Word document if the answers are text-based. Grading happens online and can be distributed among a team (instructors and TAs).

Gradescope

In addition to testing online (see below), Gradescope accepts scans of student work and uses AI to break it into questions that can be graded by multiple graders simultaneously or by the AI itself. Grading happens online and can be distributed among a team (instructors and TAs).

6. Student In-Exam Support

Let your students know how they can communicate with you if they have questions during the exam. This can be in the form of email communication or create a Collaborate Ultra/Zoom meeting where students can join the session during the exam when they have questions. From the experiences shared by instructors so far, these options seemed to work well and students appreciated knowing that the instructor was available when needed. It gives them a similar experience as a face-to-face exam where they had the opporutnity to raise their hand and ask a question.

Sources of information used here:

In addition to facts and links synthesized by the Skylight team, this document draws on:

[1] Purdue’s “Teaching Remotely” site: https://www.purdue.edu/innovativelearning/teaching-remotely/

[2] UBCO’s Teaching Remotely, Assessments: https://ctl.ok.ubc.ca/teaching-remotely/assessments/

[3] Academic Integrity in Online Learning from U. Calgary’s Taylor Institute: https://taylorinstitute.ucalgary.ca/academic-integrity-online-learning

Learning technology support

If you have any questions or if you would like to report an issue regarding a learning technology, please don’t hesitate to contact us at LT.support@science.ubc.ca. For in-person support, please join our LTRs at one of our drop-in sessions.