Strategies for Assessing Professional Judgement and other High-Level, Real-World Skills
Track: Test Development and Administration
Many test sponsors would like to assess candidates’ abilities to solve the types of problems that they experience on the job. They would like to measure professional judgement, analysis and communications skills, but they want to avoid the complexities and cost of test development and scoring of performance-based tests.
This session explore how several organizations have met these objectives by creating innovative items that address high-level skills using existing testing technologies with simplified development and scoring processes.
The presentation will show how major test sponsors use combinations of controlled, constructed-response and expanded, selected-response algorithms to create complex, job-related problems that are still easily machine scored.
The presenters will review the design, development, and scoring of: situational judgement tests, advanced-level case management problems, task-based simulations for financial analysis, simulated picture archiving and communication systems and simulated sonogram console item types —all of which are already successfully being used in national and international examinations with on-site scoring.
They will also discuss the reasons why test sponsors sought to develop these innovative items, the successes and challenges encountered along the way and the reaction of their candidates to the new items.