Situational Judgement Test

Situational Judgement Tests (SJTs) are a measurement method designed to assess individuals’ judgement regarding situations encountered in the workplace. Candidates are presented with a set of hypothetical work-based scenarios and make judgements about possible responses, which are evaluated against a pre-determined scoring key. Response instructions can be presented in either one of two formats: knowledge based (i.e. what is the best option) or behavioural tendency (i.e. what would you most likely do). SJT scenarios are based on detailed analysis of the job role, to ensure that test content reflects the most important situations in which to test candidates’ judgement.

The aims of this module of the project were to develop, pilot and validate a machine-markable Situational Judgement Test (SJT) for entry into surgical training including:

Develop and refine a bank of SJT items based on an agreed test specification
Pilot and evaluate the SJT in terms of test and item performance, including reliability, validity (content, criterion & construct), fairness and candidate reactions
Produce an initial operational paper, item bank and guidance for live test use

The project involved a series of 3 SJT pilots with candidates applying for surgical training:

Pilot 1: Initial proof of concept (February 2009)
Pilot 2: Validation alongside MRCS examinations (September 2009)
Pilot 3: Main multi-deanery pilot (Spring 2010)

The SJT report provides an overview of all phases of the project, including use of the operational SJT going forward and is structured as follows:

Introduction to SJTs and development process
Overview & results from initial pilot (February 2009)
Overview & results from validation pilot (September 2009)
Overview & results from main pilot (Spring 2010)
Development & use of operational test going forward
Summary & conclusions
References & appendices