Case Study - Medicine - Ireland

Innovative Approaches to Assessing Clinical Skills of Large Groups Using Video

Innovative Approaches to Assessing Clinical Skills of Large Groups Using Video

Background

LECTURER: Evelyn Kelleher, evelyn.kelleher@dcu.ie

DISCIPLINE: Nursing and Human Sciences

SUBJECTS: General Nursing, Intellectual Disability Nursing, Mental Health Nursing, Integrated General and Children’s Nursing

LEVEL: Undergraduate

CLASS SIZE: 232 students, from 4 pathways

MODE OF DELIVERY: Blended Learning

Overview

This case study relates to the implementation of an innovative blended learning strategy to assess clinical skills for large cohorts of students in the undergraduate BSc. Nursing in the School of Nursing and Human Sciences in DCU. The class size in 2017 was 232 divided between four pathways; General Nursing, Intellectual Disability Nursing, Mental Health Nursing and Integrated General and Children’s Nursing. I faced complex educational and logistical challenges assessing practical clinical skills for such large cohorts of students. Having reviewed the options available, I switched to a blended learning approach which included replacing paper based face-to-face practical exams with online video submission. Students buddy-up to assist each other and use their own smart devices to record the skill.

My research suggested that this would provide flexibility for students and enhance their learning, eliminate errors and inconsistency in marking, and reduce the resources and time required to run the assessment and provide feedback. The use of video in education is not a new concept, however this approach of using online video submission to replace face-to-face practical exams appears to be a new innovation.

Video: Using Video for Assessment & Feedback.

What Was The Teaching & Learning Challenge?

There were a number of teaching and learning challenges with the paper-based method:

Students had no flexibility as to when they did the face-to face exam. They were allocated a specific date and time to attend.

Pressure, nerves, anxiety and stress for students can affect or enhance their usual performance.

Fatigue and boredom can potentially affect assessor’s concentration and objectivity leading to inconsistency in marking.

The mark sheet is the only record of the student performance so there is no way to verify results if there is later disagreement.

Paperwork can go missing.

Manually entering marks on the computer system for the face to face exam left a lot of room for error.

The face to face practical exams were extremely resource intensive requiring a lot of time and human and material resources.

Few students made appointments to receive feedback so valuable teaching and learning opportunities were missed.

Addressing The Challenges

In order to address these problems, I replaced the paper-based face-to-face assessment with online video submission in 2013/2014. Students now submit a video of themselves performing the nursing skill. They buddy up and use their own devices (i.e. smart phone, IPad) to record each other performing the required clinical skills and then upload the videos to Google Classroom. The module co-ordinator and one other person independently mark each video on Loop according to a marking guide. This format is now embedded in the module and has been used each year since then with ongoing refinements.  

Identifying What Worked

This had overcome the problems identified with the paper-based face-to-face format.

It offers more flexibility as students can submit their recordings anytime within defined timespans.

Having the option to record multiple attempts reduces the negative emotions associated with having one attempt in front of an examiner. They can have multiple attempts and submit what they feel is their best attempt. This can encourage self-assessment and possibly peer-assessment.

Fatigue and loss of concentration are no longer issues as examiners can take breaks and rewind the videos. Consistency is further enhanced by having 2 markers for each video. If they have different results for a student, they can watch the video together and agree a mark.

It ensures consistency as each submission is marked independently by 2 examiners. If they have different results for a student they can watch the video together and agree a mark.

If results are disputed the student and the assessor can review the performance together to see exactly what was done.

There is no risk of paperwork going missing because all videos are marked online. This also eliminates the potential of errors inputting results.

There are significant time and resource savings

  • time required per student reduced from roughly 10 minutes to less than 4 minutes.
  • number of academic staff required from 8 to 2 over 2 days
  • no paper or photocopying needed

Online feedback is available once all submissions are marked. Students can read the feedback while looking at their video recording rather than trying to remember the exam.

In addition to these benefits this pioneering change in the assessment for this module supports the five pillars of academic integrity. Students self and peer assess videos and submit their best attempt thus giving them responsibility for their own learning. The ability for assessors to review video recordings ensures consistency and brings transparency to the assessment. Also students can revisit video recordings with feedback and see that their grade is an honest and fair reflection of their performance thus building trust and fostering mutual respect.

Tips For Implementing This Practice

Ensure there is a designated point-of-contact for student with queries or problems.

Ensure appropriate facilities and equipment are available “Bitesize chunks”Ensure appropriate facilities and equipment are available.

Have good IT support and appropriate platform to allow uploading and marking videos.

Use “Bitesize chunks”. It may be necessary to deconstruct skills to their basic components. For example, vital signs are considered one skill but students submit a video for each of its four components of temperature, pulse, respirations and blood pressure.

Give students clear guidelines in terms of:

  • what is being assessed and needs to be visible on the video
  • the technological requirements for recording and uploading the video

Consider providing a video demonstrating the application of the guidelines.

Have a marking checklist that clearly defines the skill requirements and marking structure.

Consider double-marking videos.

Reflections & Future Plans

It was hard work to introduce this new assessment format as it was a venture into the unknown that required team effort and commitment, and buy-in and support from IT Support and Teaching Enhancement Unit.

It has been extremely positively evaluated by students

      • The vast majority of them preferred the online submission format.
      • They valued the opportunity to review their recordings, have repeated attempts, and submit their best effort.
      • They reported being less stressed, anxious and nervous.
      • They liked the flexibility it offered and acknowledged the opportunity for self and peer assessment during which they could observe and reflect on their performances enabling them to identify areas for improvement and take responsibility for their own practice and learning.

I also found a number of benefits that I did not anticipate:

      • Students have started recording themselves during independent practice and reviewing the videos among themselves.
      • In cases where students challenged their results they were able to discuss application of the theory to practice thus demonstrating higher order thinking skills.
      • Students who are now doing face-to-face practical exams in other modules are requesting the option of video submission. As a result, module co-ordinators are enquiring about it.

My future plans for this module assessment include:

      • All first year practical exams are now by video submission. In response to student and module co-ordinator requests we will start rolling out to second years in 2018/2019.    
      • As more students complete the online video submission we are building a bank of data which can be analysed to compare online video submission with face to face practical exams in terms of students results and learning.
      • Video submission being used for some in service training for qualified nurses in partner services showing that this is a transferrable strategy.
Better Lecturers Copyright

You are free to share and/or adapt this material in any medium or format, but you must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. You may not use the material for commercial purposes.

Better Lecturers Dublin
Submit your case study now to help Better Lecturers get even better!