Asynchronous Item Review: An alternate method
Track: Test Development and Administration
A critical step in the test development process is developing exam content. Licensure and certification programs must ensure that their exam items have been drafted, reviewed, revised, and approved prior to being placed on an exam for pilot testing. This process is extensive but necessary, in order to build a valid and reliable exam and ultimately a strong licensure or credentialing program.
The traditional model of item review consists of face-to-face or virtual (e.g., web conference) meetings with subject matter experts (SMEs). These meetings vary in length, but may span several days or more. This model has many benefits, but it can also be time consuming and costly for programs. In addition, face-to-face and virtual meetings need to be planned well in advance to account for possible scheduling conflicts and/or weather permissions.
An asynchronous item review model counteracts some of these disadvantages of the traditional item review model. This type of model can decrease costs, allow for SME flexibility, and save time—while maintaining fidelity to the process traditionally handled during a conference-style meeting. For these reasons, the asynchronous item review process may quickly become a popular option for licensure and certification programs looking to compress the test development timeline without sacrificing the quality of the items.
How can a credentialing program use asynchronous item review to quickly and efficiently get items reviewed by multiple subject matter experts? In this session, we present the results from a program who used asynchronous item review to collect item feedback from their SMEs. We will discuss the process for facilitating such a review, the benefits and considerations of asynchronous item review, and the psychometric challenges that can be present.