Review and Evaluation of a New Recertification Assessment Model
Track: Test Development and Administration
In the US, significant physician pushback was encountered as the American Board of Internal Medicine attempted to strengthen requirements for recertification. Pushback has also been experienced from physician assistants regarding their recertification process. Now, at the state level, legislation is being introduced that undermines the role of effective recertification in protecting the public. How are certifying programs to react? One approach is to take the opportunity to explore significant recertification redesign. Can assessment programs be created that are more meaningful, more relevant to practice, and less burdensome for the practitioner?
In January 2017, the American Board of Pediatrics launched a pilot program that has been designed to incorporate a learning component into a continuous assessment model (i.e., a small number of questions administered each calendar quarter). If successful, the pilot program will eventually replace the current policy requiring certified physicians to pass a secure, proctored exam once every 10 years. During this pilot phase, data are being collected to investigate (1) the validity of using scores from this type of assessment to inform high-stakes recertification decisions, (2) the effectiveness of the learning component, and (3) the perceived value of the program by pilot participants. In this session, data from the pilot will be will be shared and discussed.
Also to be introduced and discussed is an instrument designed to facilitate critical evaluation of recertification programs. This tool provides a useful structure to help certifying organizations appraise such factors as how well program components are supported by evidence, how objective or rigorous each component is, whether each component supports or improves practitioner learning, and how relevant or related to practice each component is. The tool includes provisions to rate the burden and costs of the program to practitioners as well as administrators (i.e., certifying bodies). Program attributes to be addressed include time between “touchpoints,” continuing education targeting, continuing medical education quality and diversity, demonstration of competence and reflective practice. Evaluation criteria are grouped by 1) Validity/Public Protection, 2) Practitioner Perspective and 3) Administrator Perspective, to provide a view of the evaluated program from different perspectives.