Objective Structured Clinical Examination Nursing Essay

“The control of the licensing power is the most important function of the medical boards. Acting on behalf of the state, it is their duty to see that all candidates for license are properly qualified. They stand as guardians of the public and the profession, and here their responsibilities are indeed great.”

Osler’s statement at the Canadian Medical Association annual meeting in 1885 is still valid.

Assessment is an integral part of medical education. The primary aim of assessment is to evaluate an individual’s competence in a particular area of practise. Is this the only aim of assessment? Mackintosh and Hale (1976) suggest six possible purposes of assessment, which are: diagnosis, evaluation, grading, guidance, selection and prediction. Assessment in medical education can be formative or summative. Formative assessment helps students to develop skills and encourages learning. It is supportive and provides feedback which in turn facilitates deeper learning. The disadvantage of formative assessment is that not all students take it seriously. The summative assessment is used to judge whether an individual is competent enough to progress to the next level. It can be threatening and usually there is no feedback, however, students tend to take this form of assessment more seriously. It stimulates last minute superficial learning as opposed to the deeper learning that occurs with formative assessment. There is no single assessment tool that can reliably assess medical students. Different methods are available to assess their knowledge, skills, attitudes and professionalism. As part of this assignment I have designed two OSCE questions, which I have attached as an appendix. I will reflect on assessment methods with particular reference to the OSCE questions that I have designed.

Miller (1990) proposes a pyramidal framework for clinical assessment. The base of this pyramid represents a knowledge component (knows) followed by application of knowledge (knows how). This is in turn followed by performance (shows how) and the apex of the pyramid represents actions (does). Medical students are tested on their knowledge, application of knowledge and in vitro performance whereas work based assessment, which assesses in vivo performance, occurs after graduation and forms the final step. As a medical student my knowledge and application of knowledge was tested by written essays. Unfortunately assessment using this method is subjective. Objective approaches to test knowledge and its application include multiple-choice questions (MCQs) and extended matching questions (EMQs). Oral examinations (also known as viva voce) and long case clinical examinations were used to assess my clinical skills. Assessments of this nature are often criticised because they are unstructured and subjective. At present objective structured clinical examinations (OSCEs) form the backbone of performance assessment in medical schools throughout the United Kingdom and many other countries throughout the world.

Harden et al (1979) describe the use of Objective Structured Clinical Examination (OSCE). OSCE has changed the assessment of clinical competence because it uses actors and scenarios. In OSCE the clinical competence is assessed in a planned and structured way with attention given to the objectivity of the examination. It is a ‘focused’ examination with each station focusing on one or two areas of competence. This is a performance assessment in that it assesses student’s performance rather than their knowledge. Unlike the traditional clinical examination the objectivity of OSCE is ensured by candidates being examined by more than eight examiners, agreeing assessment criteria in advance, confronting all students with the same task, standardising patients and training examiners. The emphasis is in testing what they can do rather than what they know.

In my opinion, communication skills are essential for all doctors. History taking is a basic form of communication skill, whereas breaking bad news is a more challenging communication skill. Based on the above statement, I have designed my two OSCE questions for a history taking station and an explanation station for breaking bad news. The General Medical Council stresses the importance of good communication in its document ‘Good Medical Practice’. I firmly believe that a good clinical history and examination are crucial in establishing diagnoses and planning appropriate management. By eliciting an appropriate history, the clinician can gain insight about the illness and concerns of the patient. The art of obtaining a good history is often forgotten. Methods suitable for assessing history-taking skills are traditional long case examinations and OSCEs. Unlike OSCE, the process of obtaining the history is never observed in long case examinations. The presentation skills of the student take centre stage in the long case method. The examination format that would assess advanced communication skills, such as breaking bad news, are live observations as in OSCE or video assessment of the consultation for practising doctors.

Van der Vleuten (1996) described five criteria to assess the utility of assessment methods. Those criteria were reliability, validity, educational impact, acceptability and cost (feasibility). He defined the utility of an assessment as a multiplicative of these five factors. If any one of these values is zero, then the assessment becomes useless. For example, a test which is reliable and valid with high educational impact is unlikely to be used unless it is acceptable and feasible. There is no assessment method which scores highly in all of these components.

Reliability is the ability of a test to produce a consistent and reproducible result. Van der Vleuten (1996) suggests that tests containing a small sample of items, such as essays, stations, patient problems or tasks, produces unstable or unreliable scores, and sample size requirements vary with the efficiency of testing methods. Traditional long case and viva voce examinations are unreliable due to limited sampling. Although OSCE is an efficient assessment method the reliability is questionable for a two-station communication skills examination. As part of multi-station examination it has been shown that OSCE stations can assess history taking and communication skills with acceptable reliability (Hodges et al 1996). Obstacles to reliability could be related to the examiner, the standardised patient, or the student. Possible solutions could include training examiners, structured guidance to the examiners and proper guidance and explanation of the clinical scenario to the standardised patient.

Validity refers to the ability of an assessment tool to accurately measure the desired endpoint. In other words, to which extent the findings of the assessment are closer to the real world. Validity is a conceptual term. There are four types of validity: face validity, content validity, construct validity and predictive validity. History taking skills and communication skills are components of the medical undergraduate curriculum that ensure content validity. In communication skills testing, history taking OSCE questions could be used primarily for third and fourth year medical students, reserving breaking bad news OSCE questions for final year medical students. Construct validity refers to the ability of an assessment to differentiate between novice and expert. There are no studies that analyse the construct validity of communication skills OSCEs. There are certain factors that could adversely affect the validity of these stations. This could be related to time constraints, because in real clinical situations breaking bad news will often require more than the allotted eight minutes in the OSCE station. Hodges (2003) argues that total lack of other health care professionals in OSCE scenarios questions validity, especially because of current emphasis on multi-disciplinary health care delivery. I do not think it will be possible to generalise a good or a bad performance in particular communication skills OSCEs to predict similar behaviour in other communication settings. Colliver et al (1998) have shown that to assess empathy in a standardised patient reliably, as many as thirty-seven different scenarios could be required.

Harden (1992) has described the educational impact of assessment on learning. He used a bicycle as a model when considering the relationship between learning and assessment with the front wheel representing learning whilst the rear wheel represents assessment. Van der Vleuten (1996) has documented that assessment could drive learning through its content, format and through the feedback that follows assessment. Feedback is a feature of formative assessment. Hodges (2003) questions the validity of OSCE on the grounds that the examination itself could contribute to changes in a student’s behaviour. For example, if students knew that communication skills are tested, would their performance reflect the real life outside the examination? The educational impact of assessment is an important consideration and I personally feel that history taking skill and communication skill OSCE stations in the examination will encourages students to practice them and this will have beneficial effect in the long run.

Acceptability is an important consideration in designing an assessment method. In traditional examination methods, the process of assessment is lead by the examiner. The examiners are allowed to use their expertise. Unfortunately this makes the assessment subjective. In OSCEs the examiner marks the candidate using a structured marking sheet, which makes the assessment more objective. In history taking and communication skills OSCE stations the role of the examiner is mainly as an observer. This has led to the use of the standardised patient as assessor in the United States Medical Licensing Examination (USMLE). I was an OSCE examiner for a communication skills station few months ago and I have noticed that the standardised patient’s opinion about the performance of each candidate concurred with my own assessment. There are many studies on this topic but none of them are conclusive. The Postgraduate Medical Education Training Board (PMETB) in its consultation document stressed the importance of using lay people in the assessment process for areas of competence that they are capable of assessing. If the same principle applies to undergraduate medical education, OSCE stations like history taking and communication skills, standardised patients can be used as assessors. Further studies are needed to implement such a strategy in “high-stakes” summative assessments such as final year undergraduate medical examinations. However, in formative assessment, with feedback from students and standardised patients, this could be implemented. This will reduce the manpower required to organise the assessment and reduce the cost of running some OSCE stations.

Feasibility of an assessment method is determined by the time, cost and other resources that are required. Although it may be possible to complete a history taking or a communication skills OSCE station for undergraduate final year medical students in the prescribed 8 minutes, the same may not be possible in postgraduate settings such as psychiatric examinations. The recruitment of many highly trained standardised patients and examiners for OSCE stations involve considerable cost. The cost could be reduced by running examinations throughout the day. However this may cause fatigue and could adversely affect the reliability and validity. Other resources that may be required include a large examination hall that could accommodate ten OSCE stations and ancillary aids like an X-ray viewer.

Of the two scenarios, the history taking skill could be examined with a traditional long case method involving a lower cost to the organisers, but the explanation OSCE station for breaking bad news will certainly involve a standardised patient as the sensitive nature of the scenario will preclude using real patients.

Conclusion

Good communication skills are essential pre-requisite for all doctors. Both the OSCE questions I have designed assess medical student’s communication skills. The history taking OSCE station is designed to assess basic communication skills whereas the explanation OSCE involving breaking bad news tests more advanced communication skills. They can both be used in formative and summative assessment. They can be used to assess medical students with varying levels of experience. The reliability of two-station OSCEs is debatable. However, when used in combination with other OSCE stations in a multi-station examination format they become more reliable. They are valid when assessing communication skill for undergraduate medical students. The educational impact of communication skills OSCE stations will be positive, and I believe that this will encourages students to improve their communication skills. They are an acceptable and feasible assessment method. Two important questions that remain unanswered are: “does performance in an OSCE station predicts performance in the real world?” and: “does performance in one scenario in a communication setting generalise similar performance in other scenarios?” I would like to conclude with the words of sociologist Erving Goffman (1959) ”Life itself is a dramatically enacted thing the world is not, of course a stage, but the crucial ways in which it isn’t are not easy to specify.”