Research & Development Summit

Research Snapshot: A Look at ICE Research

Tuesday, October 24 | 2:00 pm - 3:30 pm CT

Presenters: John Wickett, PhD; Lawrence J. Fabrey, Ph.D.; J.G. Federman, Ed.D., MS, PgC; Patricia M. Muenzen, MA; John Weiner, MA; Daniel Wilson; Patricia Young, MA; Anthony Zara, PhD

Join ICE's Research and Development Committee to discuss topics, projects, and activities the committee has been pursing through the year. This is your opportunity to ask questions, provide real-time feedback, and influence the research ICE publishes.

New this year! This year’s session will be comprised of two parts: a presentation followed by roundtable discussions. The roundtables provide a deeper dive into the topics that resonate best with your interests. Roundtable topics are as follows:

ICE Online Resource Library

Please join the ICE Online Library of Credentialing Industry Research Task Force for a roundtable discussion of an exciting member benefit! The library's goal is to serve as a primary resource for credentialing organizations to share best practices, benchmarks, current research and industry trends: within a secure, accessible environment. The discussion will include submission materials, publication requirements, expectations and pricing. We welcome input from interested attendees on the development of this membership tool and expect our future Task Force activities to be shaped by the dialogue and interests of participants.

Complete the Pre-session Survey

Innovative Item Types

The innovative item type project was a collaborative project with ATP. It is composed of a white paper that provides an overview of considerations for incorporating innovative item types into assessments and an accompanying portfolio of sample items that were donated by test sponsors and vendors. During the roundtable discussion, participants can learn more about the content of this project and discuss their experiences and questions on the topic. 

International Expansion

Please join the International Expansion Task Force for a roundtable discussion of the results of the survey we conducted last year at the ICE annual meeting regarding members’ international experience and interest. The International Expansion Task Force has been formed by ICE to provide membership with information related to issues surrounding expansion of credentialing programs from domestic-only to international.  This year, we plan to refine and focus the questions and your participation will provide additional detailed data from members, like yourself, who are facing international credentialing issues. This data will help us to determine the direction of Task Force activities for 2017-2018 to develop products and information to meet membership needs.

Measuring Non-Cognitive Competence

In January 2017, the taskforce published a report to inform the ICE community on the topic of non-cognitive assessment and its relevance to professional credentialing.  The report summarizes basic concepts regarding non-cognitive characteristics, how they are measured and their potential relevance to professional credentialing and competency assurance. This roundtable session provides an opportunity to discuss the report and share information and experiences in this area. The lead author of the report will be present to answer questions regarding the document.

Recertification Benchmarking Survey

Join the Recertification Benchmarking Survey Task Force for a roundtable discussion at the 2017 ICE Research Summit. Use this time to comment on the planned data collection effort and/or identify what benchmark data you are most interested in.

Small Program Statistical Issues

The ICE R&D Task Force on Small Program Issues was established in 2016 to explore the challenges facing small credentialing programs. During the roundtable discussion, participants will have the opportunity to discuss findings from interviews conducted with individuals working with small programs on the solutions they implement to overcome these challenges. They will also have the opportunity to be part of the continuing research on the topic.  

Standard Setting

Final refinement of a draft paper on standard setting is planned soon, and the purpose of the roundtable discussion is to help ensure that relevant topics are addressed.  The purpose of this paper is to provide an introductory overview of standard setting processes, methods, approaches, challenges, issues, and policies for credentialing organizations.  Major sections of the paper include general components of standard setting, methods, policy issues, and current areas for further discussion.  These areas currently include:

  • Frequency of standard setting studies
  • Normative considerations
  • Equating vs. a standard setting study
  • Issues for a new certification program or for low candidate counts

For possible discussion:

  1. What are the other potentially controversial issues related to standard setting?
  2. What do you want to be sure is addressed in the paper?
  3. Modified Angoff appears to be most common, what other methods are your organizations using?

Validation Strategies

ICE recently published the paper on validation strategies, and the purpose was to document methods that licensure and certification programs use to establish their programs’ validity. The results verified use of job analysis as the primary validation activity conducted by organizations.  With great attention devoted to validity concepts in the past several years, the task force was somewhat disappointed that major innovations in validation activities were not revealed.  For possible discussion:

  1. What innovative ideas does your organization have related to validity?
  2. What validity claims do you make about your program? Or about your credential holders?
  3. Should ICE pursue future research on validity issues, and if so, how?

What Your CFO Wants to Know: Business of Certification Benchmarking Survey

Join the CFO Survey Task Force for a roundtable discussion at the 2017 ICE Research Summit. Use this time to get a preview of the exciting new platform for the upcoming survey that will allow respondents to benchmark their responses to those of other organizations. Participants will have an opportunity to provide input on the draft survey questions and provide feedback on the business metrics that would be most interesting or helpful to their organizations.