Procedure Based Assessments and their role in surgical training
created 01 December 2016
Procedure Based Assessments (PBAs) are a way of looking at a surgeon’s technical abilities. They can also be used to make an assessment of attitudes although this is not the main function.
The surgical Colleges have made major advances in defining the curriculum for surgical training in the ten surgical disciplines. Assessment of the outcome of training is very successful in relation to the knowledge aspects through the JCIE examinations. These exit exams are a stringent test of knowledge and higher order thinking and represent a very high standard across the world. However, the challenge is to assess a surgeon’s skills and attitudes as well as his knowledge. After all, patients are very interested in whether a surgeon is good at the procedure and they are also particularly interested in how he/she relates to them as a fellow human being.
Assessment
The term “Procedure Based Assessment” should be understood primarily as formative assessment. In formative assessment, the main aim is to give feedback which will encourage and direct learning. Summative assessment is usually an examination, is judgemental and is pass or fail. Procedure Based Assessment should not be seen in this light.
Procedure Based Assessments are one of a number of workplace based assessments which include the multisource feedback peer assessment tool (MSF), clinical evaluation exercise (CEX), case based discussion (CBD) and direct observation of procedural skills (DOPS).
For surgeons, the Procedure Based Assessment is the one that is most relevant, particularly in the later years of surgical training.
A careful study of the recording tool for the Procedure Based Assessment will show that the scoring system is very basic. The aim here is to indicate which elements of the procedure were assessed; if they were assessed whether the outcome of that process was satisfactory or unsatisfactory.
The most important part of the tool is the comments box which, on the ISCP website, expands to involve as much comment as is deemed necessary. These comments are the outcome of the discussion and they are the record of feedback that will inform the ARCP process. It is these comments which will give a Training Programme Director reassurance that good learning is being given and received.
Process
- PBAs can only be done where the trainer and the trainee are together with the patient.
- A PBA can only be completed immediately after the procedure has been completed. There is no place for a PBA form to be completed days, weeks or months after the event. It must be immediate.
- While a PBA can be carried out by a senior trainee with a more junior trainee, the only ones that count for the purposes of the training record are those that are carried out by a recognised trainer with the trainee.
- The purpose of an individual PBA is formative.
Feedback
The real strength of this process is in the feedback that a trainer gives to a trainee and the discussions that take place about the items chosen for that discussion. A collection of PBAs carried out over a period of several months can be used for summative assessment. This depends on the global summary score and whether the trajectory is from 0 towards 4. If it is, this indicates that there is good learning taking place and it can be used, along with other data in an ARCP to allow progression to the next stage of training. The concern would be where the Training Programme Director and an ARCP panel see no progress or deterioration in the global assessment scores. This is a reason for asking questions about what is happening with a trainee or indeed whether the trainer is doing a good job.
Based on my personal use of PBAs, I find them an extremely useful tool for giving feedback and I commend them to trainers and trainees. Further details on how to use them appropriately can be found on the ISCP website and also in the short PBA course run by RCSEd.
Ian K Ritchie PRCSEd, Consultant Orthopaedic Surgeon and President of the Royal College of Surgeons of Edinburgh.
Lewis Ashman
(+admin +former tickbox admin)