First Time Attendee Reception, Sponsored by Heuristic Solutions

Hosted by I.C.E.'s Program Committee, this reception is designed to encourage networking among other first-time attendees and provide an opportunity for you to meet I.C.E.'s volunteer leaders. This is an invitation-only event for first-time attendees.

Public Member Breakfast

All public members are invited to attend this breakfast hosted by the Public Member Committee. Meet your fellow public members, and identify sessions to attend at conference.

4 Ways to Grow Your Credentialing Programs

Business of Certification; Level of Content: Intermediate

I.C.E. Exchange Program Description: Due to monumental shifts in the workforce and education in response to the COVID-19 pandemic, professional learners are turning to credentials (i.e. certifications, certificates, and licenses) to upskill, pursue employment opportunities, and advance and be promoted. Due to this increased demand, organizations have been able to grow their programs in light of the challenges they have faced in recent years. With The Great Resignation comes a great opportunity for organizations to help reskill and upskill those trying to navigate their career paths.Four successful ways to grow your credentialing programs wilt be shared. Attendees will also learn about the rise of professional credentials and why it’s time to view learning and training as a way to accelerate your learning business. Leave with actionable tips to effectively grow credentialing programs and hear real-life examples of organizations that have successfully grown their programs by taking advantage of the opportunity at hand.Teaching format, methods and strategies: Traditional concurrent session with resources to support engagement.

Speakers: Ashish Rangnekar, BenchPrep, Christine Peck, CAE, ASIS International

Ethical Communications in Marketing

Marketing and Communications; Level of Content: Intermediate

I.C.E. Exchange Program Description: In this session examine the fundamental questions, "what is ethical communications in marketing?” and “Why is it so important?” Consider key issues related to ethical marketing and advertising communications, taking federal and state regulations into account. Review best practices to promote transparent, truthful, and substantiated communications on the Internet and in comparison and testimonial advertisements. Examine data brokering, outside referral sources, and the challenges and benefits of communicating via social media platforms.Teaching format, methods and strategies.: Audience Interactive Discussion

Speakers: Pete Nielsen, MS, California Consortium of Addiction Programs & Professionals

How does Longitudinal Assessment Recertification Protect the Public?

Credentialing Innovations; Level of Content: Intermediate

I.C.E. Exchange Program Description: A very promising and even revolutionary method for recertifying professionals and helping them to remain up to date in their profession is Longitudinal Assessment. This new process is purported to not only assess, but also help practitioners maintain their knowledge and skills. But not all of these new programs are the same and some include components such as article-based assessment. How are the programs different and what questions should be considered about how the processes help protect the public? When a participant performs at a low level on a longitudinal assessment could that directly lead to credential loss? How long can poor performers remain in a “remediation” status? Does your program provide for individualizing content to better reflect a practitioner’s needs or specialization? What components of your longitudinal assessment does your board consider to be of most value to the public? Seasoned practitioners who developed and implemented three different longitudinal assessment programs will respond to these and other questions and cite evidence to illustrate how their programs meet recent practitioner demands to maintain relevance, reduce practitioner burden and control the costs of recertification.Teaching format, methods and strategies.: Traditional concurrent session with moderated Q&A.

Speakers: Grady C. Barnhill, MEd, ConsultingRobert C. Shaw, PhD, National Board for Respiratory Care, Sarah Schnabel, PhD, American Board of Ophthalmology, Josh Goodman, PhD, National Commission on Certification of Physician Assistants, , , ,

Smaller Program Accreditation: The Path to Success

Standards and Accreditation; Level of Content: Intermediate

I.C.E. Exchange Program Description: Credentialing excellence is clearly demonstrated through accreditation of a program. If you are a smaller program, just getting started, or have a small pool of potential candidates, does voluntary accreditation make sense? Attend this session to hear a small program accreditation success case study for the Council for Homeopathic Certification (CHC) spotlighting small program specific challenges and solutions. Accredited by the National Commission for Certifying Agencies (NCCA), the CHC certifies individuals who meet and maintain a recognized standard of professional and ethical competence in classical homeopathy. For the CHC, accreditation was a crucial step in support of its mission. As a smaller program, CHC began the process without the resources of larger programs. Still, in 2017, NCCA granted CHC accreditation for its Certified Classical Homeopath (CCH) credential. Lessons learned, best practices, what to avoid, and how to partner will be described during the presentation. Learn about the importance of receiving NCCA accreditation and its value to CHC. Teaching format, methods and strategies.: Case study presentation

Speakers: Sheetal Tiwari, CCH, Council for Homeopathic Certification; Patricia Young, MA, Kryterion

Sunshine and Roses or Doom and Gloom? Debating the Future of Testing

Business of Certification; Level of Content: Intermediate

I.C.E. Exchange Program Description: The opt out movement is in full swing in the education space. From K-12 to admissions, critics are vocally urging leaders to forego standardized testing and their advocacy is seeing results. Today, more colleges and universities are test optional than at any time in the past. Does this foreshadow changes in the credentialing and licensure space? Is it possible that testing will become just one of several options individuals can use to demonstrate their qualifications for a job or profession? As we look to the future, how will verification of skills and knowledge will evolve - will apprenticeships and diploma privilege render testing obsolete? Or will the intersection of the worlds of academia and credentialing include more certifications as part of a course or degree requirement? And perhaps most importantly, are there steps we can take as testing professionals to make the path ahead one of sunshine and roses, rather than doom and gloom? Join seasoned professionals for a spirited debate on the future of testing. Explore these issues and leave with key considerations to help shape the journey ahead for the assessment industry. It’s a session you won’t want to miss!Teaching format, methods and strategies.: Combined debate and panel discussion.

Speakers: Rachel R. Schoenig, JD, Cornerstone Strategies, LLC; Donna McPartland, JD, FIP, CIPM, CIPP/US, McPartland Privacy Advising; Stephanie R. Dille, MBA, Meazure Learning; Kellie Early, JD, Law School Admission Council (LSAC); Isabelle Gonthier, PhD, ICE-CCP; PSI Services

We know it’s important but what can we do about it? Considerations and guidance on how credentialing bodies can put DEI principles into practice.

Business of Certification; Level of Content: Introductory

I.C.E. Exchange Program Description: Diversity, Equity, and Inclusion (DEI) are important inter-related principles that are rightly receiving more attention now than ever. Recognizing that each individual and each organization is on a different step of their DEI journey, we must turn our attention to tools and resources available for those who wish to advance and advocate for DEI. In this session, members of the I.C.E. DEI Task Force present an introduction to DEI and three resources they’ve developed or identified for credentialing bodies: DEI End Statements, some key DEI concepts and terms, and a DEI Maturity Model. The DEI End Statements show how I.C.E. has put DEI into practice as a strategic guide.The primer on DEI concepts and terms can help the recognition and identification of the many human characteristics that can contribute to DEI. The DEI Maturity Model can help organizations identify appropriate next steps toward advancing and organizing DEI efforts, showing how DEI can start with legal and regulatory compliance, move to a more programmatic or tactical level, then on to business strategy, and finally toward organizational culture.Teaching format, methods and strategies.: Panel Discussion

Speakers: Amin Saiar, PhD, Board of Pharmacy Specialties; Amine Issa, GPHR, PHR, HR Certification Institute; Denise Roosendaal, FASAE, CAE, Institute for Credentialing Excellence; Tracy Lofty, MSA, CAE, International Board of Heart Rhythm Examiners

We Want You! Attracting outstanding SMEs

Test Development and Administration; Level of Content: Intermediate

I.C.E. Exchange Program Description: Where would we be without our Subject Matter Experts (SMEs)? The NCCA Standards devote an entire Standard to panel composition. But our potential SMEs are busy people with competing demands for their time. How do we get their attention and convince them to participate in exam-related activities? We know the work is difficult, but rewarding. Come listen to a panel discussion to hear about challenges some successful certification programs have faced and strategies that have worked well for them. Come willing to share your own stories as well. This engaging discussion will give you a variety of ideas to think about adapting to your own needs.Teaching format, methods and strategies.: Panel discussion: we are working for the panel to have a somewhat conversational tone to keep the session engaging, but the intent is for each panelist to touch on challenges they've had in recruiting SMEs and how they've met those challenges. We intend to include ample time to invite audience members to do the same: share challenges and what they've done. By hearing a diversity of approaches audience members will be able to "pick and choose" ideas that might work for their organizations. Thank you to our Test Development and Administration Track sponsor Lineup

Speakers: Daniel H. Breidenbach, PhD, ICE-CCP, PSI; Annette M. Bush, MBA, BSN, RN, OCN®, ICE-CCP, Hospice and Palliative Credentialing Center (HPCC); Kevin Letz, American Academy of Nurse Practitioners Certification Board; Nancy B. Grittman, MBA, American Association of Veterinary State Boards; Sheryl Traficano, MBA, CAE, Certification Board for Diabetes Care and Education, Shantal Johnson; National Board of Public Health Examiners (NBPHE)

Evaluating the adaptation and implications of oral exams switching from in-person to virtual format

Credentialing Innovations

I.C.E. Exchange Program Description: In the face of the COVID-19 pandemic many boards faced the obstacle of how to safely certify candidates. This was a particular concern for boards with an oral component to their exams. Several boards worked to adapt their exam format from an in-person model to a virtual one. Lessons and implications from this adaptation could impact and shape the future of how oral exams are administered. This poster aims to highlight some of the key takeaways with regards to exam development and administration, implications, and exam security.

Teaching format, methods and strategies: E-Poster Meet the Author

Speakers: Hally Crump, MA, Data Recognition Corporation

Validation Evidence for Exam Administration Modalities: Monitoring Scores Between Live Remote Proctoring and Computer-Based Testing

Credentialing Innovations

I.C.E. Exchange Program Description: Live remote proctoring offers candidates with a new, flexible alternative to traditional in-person, proctored testing. However, testing programs must provide validation evidence that these exam administration modalities are comparable. This requires an analysis of both candidate scores and item performance. This poster presents statistical analyses and psychometric considerations when providing support for the use of both exam modalities. If statistical differences exist between modalities, steps to address these differences are shared. 

Teaching format, methods and strategies: E-Poster Meet the Author

Speakers: David Chen, PhD, and Jackie Holt, MS, Scantron; Todd Galati, MA, American Council on Exercise; Natalie Rose, MHA, CPMSM, CPCS, Certification Commission for the National Association Medical Staff Services; Meghan Leeming, MS, University of North Carolina at Greensboro and Scantron

Post-Pandemic Credential Growth Model

Beyond Traditional Credentials

I.C.E. Exchange Program Description: This ePoster session highlights a novel credential growth model for post-pandemic credential operations. A balance between supply and demand-side linkages is included. Naturally, insights on credential acceleration are provided for key decision-makers to maximize organizational profitability and leverage certificant interest.

Teaching format, methods and strategies: E-Poster Meet the Author

Speakers: JG Federman, MS, Creighton University

Rocking the RFP: Optimize your RFP to elicit better proposals

I.C.E. Exchange Program Description: Executing a well-executed Request for Proposal (RFP) is an important step toward an exciting new phase for your program. Making a change to your certification program can feel challenging and risky. Authoring a clearly written and detailed RFP) will determine the success of the implementation of your new or modified certification program. This poster will provide  what to consider in your pre-planning phase as well as how to author your RFP, and what to avoid in the document and procurement process with partners and vendors. 

Teaching format, methods and strategies: E-Poster Meet the Author

Speakers:David Bennett,Pearson VUE

Test taking strategies and testing time: When is enough enough?

Test Development and Administration

I.C.E. Exchange Program Description: What test-taking strategies help most when taking high-stakes assessments? Should candidates spend more or less time on an assessment? Some candidates will use all available time when taking an assessment, while others will complete an assessment without second guessing their selves. More common test taking strategies are eliminating likely incorrect options and leaving no items blank. Some testing-taking strategies make it difficult for certifying bodies to determine if candidate have sufficient time. This poster shares information on the relationship between testing time, test taking strategies, and exam performance. 

Teaching format, methods and strategies: E-Poster Meet the Author

Speakers: James Hellrung, MA, Kryterion; Neil Guman, MA, National Center for Competency Testing

Building a compelling credentials campaign for a global audience: A Google Cloud marketing case study

I.C.E. Exchange Program Description: Creating a sticky credential activation campaign can be a complex and overwhelming task. Determining your target audience, to understanding their motivations, and ultimately crafting a compelling offer can make or break the campaign. And, what if your campaign falls flat? How do you pivot and learn from your mistakes? Review a credential marketing campaign case study, providing a closer look at the successes, the struggles and the roadblocks that informed the framework for Google Cloud’s scaled creative credential activation campaigns.

Teaching format, methods and strategies: Lightning Learning Session

Speakers:Mara Soss, Google Cloud; Heather McCormick, Google Cloud Certifications and Skill Badges, Google Cloud

Digital Accessibility: What does that mean for certification design and delivery?

I.C.E. Exchange Program Description: Understanding how disabled people use assistive technology and how that impacts our creation of content for certification exams and exam delivery starts with understanding digital accessibility. Learn how to create accessible content and evaluate systems and processes for disability inclusion.

Teaching format, methods and strategies: Lightning Learning Session

Speakers: Samantha Evans, CAE, IAAP

Herding cats: Tips on how to build stakeholder alignment and participant awareness in large scale studies

Marketing and Communications

I.C.E. Exchange Program Description: Frustrated with the lack of responses from a job analysis survey? Having a difficult time managing stakeholders’ expectations and gaining support for changes to your program? This session will highlight ways in which one organization kept key stakeholders aligned, and was able to engage over 10,000 participants in a programmatic research study involving a variety of qualitative research methods, multiple surveys, and focus groups. Learn tips and tricks on how to maximize participation with your study, plus, how to manage internal politics and expectations.

Teaching format, methods and strategies: Lightning Learning Session

Speakers: Amanda Wolkowitz, PhD,  Alpine Testing Solutions, Inc.; Vanessa Williamson, CAE, NCARB

Lock It Down: A Secure Browser POV

I.C.E. Exchange Program Description: Secure Browsers, also known as lockdown browsers, are web browsers that do not have any of the controls typically seen in a web browser (back, forward, address bar, etc.). The browser only displays the test content, and access to any other website or application not required for testing is blocked. This allows the test taker to focus only on the test content and prevents them from accessing external resources. Secure Browsers are great tools for enforcing security during a test, but there are misconceptions that they can block all attack vectors. The Secure Browser has limitations that mainly get exposure during remote testing scenarios where the candidate controls the test environment. Learn best practices in implementing a secure browser and how it can be used effectively in addition to a data forensics and candidate accountability strategy as part of an overall strategy to enforce test security. 

Teaching format, methods and strategies: Lightning Learning Session

Speakers: Pat Hughes, Internet Testing Systems; Bryan Friess, Pearson VUE

Adding ISO/IEC 17024 Accreditation to Your NCCA-Accredited Program

I.C.E. Exchange Program Description: Attend this discussion to learn more about the ISO/IEC 17024 Standards and learn considerations when seeking accreditation to the standard.

Teaching format, methods and strategies: Audience interactive discussion.

Speakers:Dr. George Anastasopoulos, International Accreditation Service

Does your Organization have a Legally Defensible Disciplinary Process?

I.C.E. Exchange Program Description: How much thought have you put into your organization’s disciplinary processes? Do your procedures provide certificants with adequate due process? Is your disciplinary review committee making legally defensible decisions? Are your committee members adhering to the organization’s written rules and procedures? Legal challenges to an organization’s disciplinary decisions can be costly. Even more concerning is the damage these challenges can do to an organization’s reputation and the value of its credential. Therefore, it is essential that your disciplinary process is structured and administered properly. This session will discuss the legal concepts every organization should consider when creating its disciplinary process. Additionally, it will set forth ways to ensure your program provides sufficient due process to certificants. Presenters will also discuss the proper way to administer a disciplinary program so that decisions are consistent and legally defensible. They will set forth steps all organizations can take to prevent arbitrary and capricious decision making. Finally, session presenters will address legal issues that may arise during the complaint process, including breaches of confidentiality and conflicts of interest.

Teaching format, methods and strategies: Traditional Concurrent Session and Q&A

Speakers: Katharine Meyer, ESQ, American-Speech-Language Hearing Association; Richard Bar, Esq., GKG Law, P.C

Job Analysis: Tips for Crafting an Exemplary Study

Test Development and Administration; Level of Content: Intermediate

I.C.E. Exchange Program Description: Practice analysis studies appear to be deceptively simple investigations—just identify and validate the domains and tasks that comprise practice, as well as the prerequisite knowledge and perhaps skills. However, with some strategic enhancements your organization's investment in a practice analysis can move it from the minimum requirements to a study providing rich returns. This presentation will provide tips on how to enhance your practice analysis, moving it from a plain vanilla study to a bespoke study that answers questions well beyond the development of a validated test content outline.

Teaching format, methods and strategies: Traditional concurrent session

Speakers: Matthew Scaruto and Jesse Lobbs, Ascend Learning; Marlene Pratt, National Healthcareer Association; Carla M. Caro, MA, Credentialing Specialist, ACT

Our SMEs are the Bee’s Knees: Three approaches to developing strong item writers and reviewers from your subject matter expert pool

I.C.E. Exchange Program Description: Any given item can go through round after round of editing and still not be quite right. So how do you go about limiting the number of edits needed when authoring new items? Join this session for an interactive discussion around recruiting, training, and retaining subject matter experts. ACE, ACMPE, and WOCNCB will share their approaches to maintaining a top-notch SME pool. Applying one or all of their strategies can save time, money, and the headaches that accompany endless review cycles. Additional topics of discussion will include how to incentivize subject matter experts when compensation is not an option, how to keep SMEs engaged during long remote meetings, and how to meet a program’s item development count goals and deadlines.

Teaching format, methods and strategies: Audience Interactive Session. Ppanelists will share strategies to SME recruitment and training. Attendees will be encouraged to share what they have done successfully at their organizations. Together, brainstorm existing (and hopefully new) ways to find, motivate, and retain SMEs long term.

Speakers: Sarah Mastrianni, Scantron Corporation; Alexandra Link, MS, American Council on Exercise; Nikki Klaus, Medical Group Management Association (MGMA); Cindy Lemek, CAE, Wound Ostomy Continence Nursing Certification Board (WOCNCB)

Practical Steps to Enhancing Diversity, Equity, and Inclusion in Your Certification Program

Business of Certification; Level of Content: Intermediate

I.C.E. Exchange Program Description: You know that assessing and addressing issues of diversity, equity, and inclusion in certification programs has become a priority for testing programs, given recent world events and the pandemic, which have once again highlighted the differences in access to resources and opportunities. And you understand that a bias review panel and differential item functioning analyses are no longer sufficient evidence of addressing fairness in a testing program. But how can your certification program evaluate your assessment and measurement practices from an inclusive perspective in order to provide evidence of an equitable assessment system? Based on the work that is being done at the National Board for Professional Teaching Standards, this session aims to show credentialing professionals a starting point and concrete examples of action taken toward enhancing diversity, equity, and inclusion in the assessment process of teaching professionals. Learn practical steps to examine the main areas of your testing program that need to be evaluated; hear concrete examples used for evaluating and making improvements; and apply shared strategies for enhancing diversity, equity, and inclusion in design, development, and operations. Leave with concrete examples of auditing for diversity, equity, and inclusion, standards revision, scoring personnel, scoring training, research, outreach, candidate experience, assessment publications, and more.

Teaching format, methods and strategies: Traditional Concurrent Session

Speakers: Andrea Hajek, NBCT, and Carol Ezzelle, PhD, National Board for Professional Teaching Standards

Raising Your Exam Security Game

I.C.E. Exchange Program Description: Cheating and content theft can seem like a never-ending cat-and-mouse game for global, high-stakes programs. Online exam security is critical to prevent proxy testing, identify potential misconduct, and protect content from being compromised or shared, but security doesn’t begin and end with the proctor. To ensure exams are protected, consistent and innovative security processes must be used and evaluated for effectiveness. Collaborating with several responsive and forward-thinking exam security providers, MuleSoft deployed a constant monitoring and enhanced security strategy that allows program leaders to not only catch “cheaters” but also identify integrity breaches and continually develop new tactics.  Hear a case study in which Meazure Learning’s enhanced security program detected data points identifying potential collusion, which led to a deeper analysis by Caveon’s data forensics team. We’ll state the analysis outcomes as well as MuleSoft’s response and policy changes given the advancements in available detection tools. Hear firsthand how the collaborative approach between three industry leaders provides a balance of security and IP protection while also focusing on the overall candidate experience. These organizations work closely to identify security threats and create scalable solutions with highly defensible outcomes. Attendees will get a peek behind the enhanced security curtain and a broader understanding of the importance of partner collaboration when it comes to protecting global exam programs.

Teaching format, methods and strategies: Traditional concurrent session.

Speakers: Gabriell Darby and Cory Clark, Meazure Learning; Simon Welham, Mulesoft; Alison Kramer, Caveon Test Security

Small but Mighty: Combining smaller assessments to enhance overall reliability

Beyond Traditional Credentials; Level of Content: Intermediate

I.C.E. Exchange Program Description: In a fast-paced world of sound bites how do assessments remain relevant? How can they be short enough to be consumable by candidates, and yet have sufficient reliability to support appropriate uses and interpretations of scores? It is generally accepted that, all things being equal, the smaller the number of items/points the lower the reliability of the assessment. What if multiple smaller assessments were combined? Would the reliability of the combination be sufficient to make inferences about what a test-taker knows and can do? This session explores these questions in depth. F5 Networks is currently moving from a traditional certification program to a more nimble program. The new program is comprised of smaller assessments that must be combined in order to earn a credential. This session presents analysis results related to the reliability of smaller assessments and how the combination impacts the overall reliability of a larger (combined) credential.

Teaching format, methods and strategies: Traditional concurrent session

Speakers:Jill Van Den Heuvel, PhD, ICE-CCP, and Jeff Kelley, PhD, ICE-CCP, Alpine Testing Solutions; Heidi Schreifels and Ken Salchow, DBA, F5 Networks

Trends in Governance: Key Findings of an Industry Survey of Governance Models

I.C.E. Exchange Program Description: Credentialing programs take various approaches to building organizational structure and selecting models for efficient and effective governance. Taking the time to understand governance best practices and benchmark against other credentialing organizations aids in selecting the right model. In 2022, SeaCrest conducted a survey to gather data and feedback on program governance issues . The 2022 project is the third version of the survey, and the results will be presented with a focus on identifying best practices along with trends and differences from previous years. The survey addresses the following topics specific to certification governing bodies: • Incorporation status • Composition & terms • Selection & criteria • Meetings • Training, orientation, & evaluation • Financial management • Quality improvement & policy review • Strategic development Results of the 2022 survey will be presented with a summary of trends and differences from previous surveys administered in 2013 and 2008. Survey results will be shared with conference attendees.

Teaching format, methods and strategies: Traditional concurrent session

Speakers: Janice Moore, ICE-CCP, SeaCrest; Sheri Sesay-Tuffour, PhD, CAE, Pediatric Nursing Certification Board, Inc.

Beyond the Panel Meeting: How Credentialing Organizations Can Use Additional Data Points to Expand the JTA and Add Validity Evidence to their Program

I.C.E. Exchange Program Description: Many programs rely on the traditional JTA panel meeting and survey to develop the list of job tasks, knowledge, and skills that form the content outline on which the examination is based. This session will encourage programs to examine additional data sources to guide and inform the job analysis and construct development process. The session will provide multiple perspectives on available technologies and techniques currently in use by two different programs to develop a stronger evidence base to the process of content and construct definition. Specific implementation of the technique in regard to development of infrastructure and training of staff/SMEs, changes to operational elements of the test development life-cycle, and reconceptualizing psychometric processes in the light of statistical process control will be discussed at an approachable level for all audience members. Lastly, the presenters will describe the process of construct mapping integrated into the JTA process. Moreover, the presenters will discuss ways ways these data sources can be used throughout the examination development lifecycle to enhance validity in a systematic and structured way. The audience will leave the session feeling informed as to how to start asking the important questions about how evidence that currently exists, or can be easily obtained, can inform a transition to a future state of better-informed JTAs and more practically meaningful validity arguments.

Teaching format, methods and strategies: The session will encourage the active participation of attendees through the encouragement of questions and discussion and the use of polling to gather information on current methods of gathering job analysis and construct definition information.

Speakers: Nadine L. McBride, PhD, National Registry of Emergency Medical Technicians; Matthew Burke, PhD, Amazon Web Services; Shonai Someshwar, MA, MSc,University of North Carolina at Greensboro

Extending Collusion Analysis to New Item and Exam Formats

Security, Records, and Data Management; Level of Content: Advanced

I.C.E. Exchange Program Description: The validity of exam results requires that performance relies on a test taker's legitimate knowledge of a domain. Pre-knowledg of test content and test results from proxy test takers are serious issues within the credentialing community. While significant work has been done to understand and detect cheating on standard multiple-choice exams, significantly less is known about alternative item types and scoring options. This session will explore issues of data forensics unique to polytomously scored items pool-based exam delivery.

Teaching format, methods and strategies: Traditional concurrent session

Speakers: Kirk A. Becker, PhD, Pearson VUE; Liberty Munson, PhD, Microsoft

From Braille to Bluetooth: Adjusting for evolving accommodations

I.C.E. Exchange Program Description : A lot has changed in the last two years, especially in terms of test accommodations. As programs increasingly pivot to online testing, we must figure out how to balance accommodations and test security. What are your creative solutions for requests to test online with complex accommodations, like using readers/recorders or testing in alternate formats like Braille? How will you handle medical equipment, like glucose meters and hearing aids, that has blue-tooth capabilities and cellphone requirements? And what about new disabling conditions caused by COVID-19, like brain fog and long COVID? The accommodations field is rapidly evolving, and we’ll navigate it together. Attend this session to discuss your experiences assisting candidates in this changing environment and learn from policies and approaches other programs have implemented. Other trending topics will be added to the discussion as they arise.

Teaching format, methods and strategies.: This audience interactive roundtable will be a facilitated discussion. The moderator will encourage participants to share their experiences and ask questions of other programs to help solve the accommodation issues they are experiencing.

Speakers:Debbie Bergtholdt, Pearson VUE

Hard Facts About Soft Skills

Test Development and Administration; Level of Content: Intermediate

I.C.E. Exchange Program Description: Dramatic and accelerating changes in work and workplaces are elevating the importance of noncognitive competency assessment vis-à-vis traditional assessment of knowledge, skills, and abilities. Technological changes, including automation, are transforming the nature of work. Growing appreciation for the importance of communication and teamwork pulls in one direction while the ballooning phenomenon of remote work demands a different set of adaptations. This session will explore in depth the rapidly evolving trends, applications, and considerations for noncognitive assessment in credentialing. An expert panel will discuss key topics related to assessment of noncognitive competencies.

Teaching format, methods and strategies: Panel discussion, on questions probing why non cognitive skills assessment in credentialing is important for public protection, for meaningful recertification decisions, and for the credibility of credentials.

Speakers: Rebecca LeBuhn, National Board of Certification and Recertification of Nurse Anesthetists; John Weiner, MA, PSI Services; Ibi Layi-Ojo, MS, SPHR, SHRM-SCP, ICE-CCP, Certified Financial Planner Board of Standards, Inc.; Chad Buckendahl, PhD, ACS Ventures, LLC

Instructionally Sound Certificate Study Prep

I.C.E. Exchange Program Description: What is the best way to develop certification review courses and study prep? By utilizing the principles of adult learning, and ensuring the offering(s) are appropriate for the delivery methods (virtual classroom, face-to-face, eLearning, or a hybrid), the needs and preferences of candidates, and the scope and intent of the certification. Can you do that while honoring firewalls? You bet, and this session will show you how, with ideas for effective study prep design and real world examples from an association who integrated instructional design in their study materials.

Teaching format, methods and strategies: Traditional concurrent session

Speakers: Michelle Nolin, CPTD, Learn Ethos LLC

Transitioning Diverse Test-Taker Populations to Online Testing

I.C.E. Exchange Program Description: Over the past two years, the credentialing industry experienced rapid growth of online testing. Many programs were forced to quickly pivot the delivery of their exams online to continue providing their credential(s). For some, this meant transitioning candidates from test centers to at-home testing. For others, it was moving from in-person or classroom testing solutions to online solutions. Many were challenged even further by test-taker populations not accustomed to using computers for their day-to-day work or that may not have access to adequate technology. In addition to test-takers, other stakeholders are impacted by these changes. Despite these challenges, many credentialing programs quickly introduced a new exam delivery modality to a wide variety of stakeholders. And like with any new challenge, learned a lot. This session will highlight the experience of two large, high-stakes testing programs that have transitioned diverse test-taker populations to online delivery. Hear about the importance of a thoughtful communications plan to introduce a large program change to all stakeholders. Understand what to consider about your audiences if you have recently introduced online testing or plan to in the future. Examples of successful communications tactics for test-takers, educators, and board members will be shared, along with the results (what did and didn’t work). Throughout the session, the presenters will highlight their experience educating a diverse population of test-takers. Hear some concrete ways to communicate a transition online to non-technical test-takers, and review the importance of test-taker choice and feedback as part of the transition to online delivery.

Teaching format, methods and strategies: Traditional Concurrent Session.

Speakers: Stephanie R. Dille, MBA, Meazure Learning; Larry Lynch, FASAE, CAE, National Restaurant Association; Joel Oliva, ICE-CCP, National Commission for the Certification of Crane Operators (NCCCOS)

Why Accessibility isn’t Just About Meeting Legal Requirements

Business of Certification; Level of Content: Intermediate

I.C.E. Exchange Program Description: Accessibility generally refers to the design of products, devices, services, programs, systems, and environments to make sure that individuals with certain physical limitations are able to meaningfully participate despite any limitation. Providing accessibility is a legal requirement in certain situations, but this session will discuss other important considerations that go beyond the legal requirements.

Teaching format, methods and strategies: Traditional concurrent session with interactive audience discussion

Speakers:Mark Franco, JD, ICE-CCP,  Whiteford, Taylor & Preston, LLP; Samantha Evans, CAE, IAAP

Changing Up Your Assessment

Test Development and Administration; Level of Content: Intermediate

I.C.E. Exchange Program Description: Learn from a membership organization as it moves three board certifications from a portfolio-based evaluation to an exam-based assessment with the help of their vendor. This session will provide two perspectives on moving portfolio-based evaluations to exam-based assessments for advanced practice certification programs. The session will include benefits, considerations, and lessons learned, while going through the program transition.

Teaching format, methods and strategies: Presentation with Q&A

Speakers:  Kate P. Kahoa, M.d, PSI Services; Teresa Dufeny, PhD, OTR/L,AOTA

Hidden Biases in your Credentials: How to maximize fairness in your programs

Beyond Traditional Credentials; Level of Content: Advanced

I.C.E. Exchange Program Description: Credentialing and assessment exams are notorious for perpetuating unintended unfair impediments that prevent access and opportunity for traditionally underrepresented identity groups. A panel of experts will examine and discuss important perspectives to keep in mind along the journey toward advocating for a much-needed transformation and reform of credentialing practices.

Teaching format, methods and strategies: Panel Discussion

Speakers: Chris Beauchamp, PhD, Meazure Learning; Adrienne Cadle, PhD, Professional Testing, Inc.; Guillermo Ortiz de Zarate, Lineup; Nicola J. Wright, CAE, Board of Canadian Registered Safety Professionals

Candidate Experience in Online Testing: Measurement Issues, Research, and Implications for Practice

Test Development and Administration; Level of Content: Intermediate

I.C.E. Exchange Program Description: Online remote administration of high stakes credentialing examinations is an existing practice that was accelerated by the pandemic, as organizations were required to quickly adapt and implement safe and socially distanced practices. During this time, organizations developed and refined remote online test delivery methods, and many have now established multi-modal test administration practices that include both test centers and remote online proctored testing. One area for consideration in these multi-mode testing programs is the potential for variability in testing environments, including the technology available, the proctoring methods, and the ability to resolve any technology issues that arise during a testing session. Any of these factors may impact the test takers’ experiences and test performance. Research in this area is emerging and offers approaches to monitoring and measuring these factors to assess potential impact on perceptions and performance, and suggests areas where testing administrators will need to prepare action plans. This session explores these issues, early research findings, and recommended practices for monitoring and measuring candidate experience, environmental considerations and technology effects to assure the integrity of testing programs. A diverse panel of measurement experts will share their experiences and research in high stakes testing in a range of testing applications and will discuss considerations and implications for operational credentialing exam program management.

Teaching format, methods and strategies: The session will follow a modified panel format with an overview presentation in the first half to set the stage for the panel to discuss key issues and considerations during the second half of the session. Session attendees are encouraged to participate in the discussion to share perspectives and engage with the credentialing community.

Speakers:  John Weiner, MA, PSI Services; Tim Muckle, PhD, NBCRNA; Maxim Morin, PhD, Medical Council of Canada; Andrew Wiley, PhD, ACS Ventures

Cheaters Gonna Cheat: Stories from Test Centers & Online Proctoring Experts

Security, Records, and Data Management; Level of Content: Intermediate

I.C.E. Exchange Program Description: Have you ever wondered how test-takers try to cheat on exams? No test – regardless of modality – is free from cheating attempts. Each has its own security concerns that need to be mitigated. And it’s important for credentialing professionals to understand the security landscape of their program so they can confidently collaborate with their partners. Join two leaders in test security as they recount real stories and examples of cheating attempts in test center and online proctoring environments. Some examples are creative, some are funny, but all of them are interesting. In this interactive session, you will learn about test security threats and proven tactics to mitigate those threats, no matter the modality. You’ll also get expert recommendations on how to conduct a health check of your program’s security landscape. You’ll leave the session with a deeper understanding of why test security should be top of mind and actionable steps to take back to your own exam program.

Teaching format, methods and strategies: Each presenter will present a specific perspective, test centers and online proctoring. After examples are shared, the focus will shift to program level tactics and assessment strategies.

Speakers:  Cory Clark, Meazure Learning; Alana Chamoun, CSBS

Innovative assessment strategies: Are they accessible to candidates with disabilities?

Test Development and Administration; Level of Content: Intermediate

I.C.E. Exchange Program Description: Innovative credentialing programs consistently review and evaluate the effectiveness of assessment design. Credentialing organizations have developed and implemented innovative assessment strategies, including unique item types, simulations, gaming, performance testing, and multi-modal forms of assessment. Indeed, the industry has witnessed a myriad of technological advances which have evolved test development and delivery to mirror the knowledge, skills, abilities, and attributes of candidates in their job roles. But while these advances are exciting to test developers, they may be problematic for candidates with disabilities. Can we develop innovations in our test design and test delivery, while still accommodating candidates with disabilities who have unique accessibility needs? This session will focus on key areas of consideration for test developers as they pursue new and innovative assessment content and methods for delivery. The presenters will address types of test adaptations that might be problematic for candidates with various types of disabilities. The presenters will emphasize practical considerations for accessible test development, and strategies to avoid potentially costly and time-consuming remediation efforts if your organization develops a test that isn’t accessible.

Teaching format, methods and strategies: Panel discussion.

Speakers:  Maria Incrocci, PhD, RPh,Scantron; John Hosterman, PhD, Paradigm Testing, LLC; Bill West

Measuring the Value of Certification: Insights and Lessons Learned

Marketing and Communications; Level of Content: Introductory

I.C.E. Exchange Program Description: This presentation will share insights and lessons learned from one organization's value of certification study. The National Board for Certification in Occupational Therapy (NBCOT) designed a study to gain insight into certificants’ and employers’ perceptions of the benefits of certification and certification renewal. Learn how NBCOT surveyed certificants to assess their attitudes toward certification, including both intrinsic (e.g., a sense of pride) and extrinsic (e.g., professional recognition) factors, and designed a survey specifically for certificants who had let their certification lapse and then reinstated it. Finally, learn about how a database of employers was created and surveyed about their awareness of NBCOT certification and their perceptions of the value of certification among their employees. In addition to learning about the challenges and opportunities that arose while conducting this study, hear the results of the study and discuss how these types of results can be used.

Teaching format, methods and strategies: Traditional concurrent session

Speakers:  Sarah Carroll, PhD, National Board for Certification in Occupational Therapy

Reimagining Your Certification Exams: Considerations, Pitfalls, and Stories from the Frontiers

Credentialing Innovations; Level of Content: Intermediate

I.C.E. Exchange Program Description: Certification bodies frequently navigate competing goals and pressures when developing and maintaining their exams – from identifying and addressing the needs of their constituents, marketing their offerings, and ensuring they adhere to test development and psychometric best practices. Similarly, once certification exams are established, updating and refining them and keeping pace with workforce requirements presents yet different challenges. During this panel discussion, gain insight into the strategic planning, innovations, and obstacles faced by three different certification bodies currently working toward change to better serve their stakeholders. The Medical-Surgical Nursing Certification Board (MSNCB) will discuss their progress in developing an exam aligned with a newly developed competency framework tailored to their nursing population. Hear the steps involved in assessing the need for multiple levels of certification and the work required to appropriately define the knowledge and skills (at each level) required by their profession. The American College of Veterinary Internal Medicine (ACVIM) will discuss their motivation to revamp the format for five specialty exams to better align content needs with psychometric standards while balancing the expectations of their board and candidates. And finally, the Certification Commission for Healthcare Interpreters (CCHI) will discuss their process of defining a need and then designing an innovative exam to fill a gap between two existing credentials. Learn from the experiences and recommendations of the panelists' journeys to innovate and best serve their professional communities.

Teaching format, methods and strategies: Panel discussion

Speakers:  Charis Walikonis, MA and Alyssa Rulf Fountain, PhD, (Moderator), Prometric:; Natalya Mytareva, M.A., CoreCHI™,Certification Commission for Healthcare Interpreters; Terri Hinkley, EdD, MBA, BScN, RN, CAE, Medical-Surgical Nursing Certification Board

State Legislation that would Constrain Enforcement of Conduct Codes and Eligibility Requirements by Licensing Boards and Certification Programs

Business of Certification; Level of Content: Intermediate

I.C.E. Exchange Program Description: Should state legislatures restrict occupational licensing boards and private certification organizations from denying a license or certification based on past criminal conduct? Can public speech contrary to consensus values of a profession be the basis for discipline by a certification organization or licensing board? State legislators have introduced and, in some cases, successfully passed legislation that limits the authority of licensing boards to consider or take adverse action based on certain criminal records. Some legislation has also sought to restrict the rights of certification organizations to adopt and enforce elements of eligibility standards and conduct codes. This session will describe a range of recent legislative initiatives under this rubric. Bills discussed will include:Efforts to expand opportunities for ex-offenders by restricting the kinds of criminal conviction history licensing boards can consider;Legislation of a “clean slate” by automatically sealing criminal conviction records after a period of time has elapsed, preventing access to those records even by licensing boards; Legislation restricting denials of certification by private organizations based on certain criminal record history; Efforts to restrict licensing boards and certification organizations from taking adverse action based on the public spread of misinformation or disinformation, and Legislation restricting certification organizations from enforcing DEI codes. The session will also discuss broader implications for certification organizations, including First Amendment considerations and protecting the self-regulation of private certification organizations. The session will pose questions about what kinds of conduct not directly related to professional practice a certification organization can or should consider in its ethics enforcement, such as comments on social media, attendance at white supremacist rallies, or spreading of COVID-19 misinformation or disinformation.

Teaching format, methods and strategies: Traditional concurrent session

Speakers:  Julia Judish, JD, Pillsbury Winthrop Shaw Pittman LLP; Tom Granatir, American Board of Medical Specialties

When you say test center, what kind of test center do you mean?  Evaluating test centers and their respective features and tradeoffs

Test Development and Administration;>Level of Content: Intermediate

I.C.E. Exchange Program Description: It’s clear that the COVID-19 pandemic disrupted the prevailing mode of high-stakes testing with the necessity of quarantines, social distancing, and lockdowns, all of which impacted the availability of brick-and-mortar test centers. Over the past two years, many test sponsors have sought alternative delivery methodologies, beyond traditional test centers. Remote online proctored testing existed prior to the pandemic, of course, but its growth and acceptance has accelerated significantly in just the past two years. As programs continue to explore the viability of online proctoring for their specific exams and credentials, much consideration has been devoted to the different features and capabilities of remote online proctoring platforms and tools and how they compare with the generic notion of traditional “test centers”. However, a weakness with those comparisons is that many in-person brick-and-mortar test centers have different features, advantages, capabilities, and risks. So, when you say test center, what do you mean? This session will explore the significant and meaningful variations between different kinds of test centers and candidate populations, including: the continuum of access and control, candidate experience and exam delivery, and security frameworks and protocols. Leave with an understanding of the tradeoffs between the options for test center delivery and be better prepared to decide the appropriate modality for their credentialing program.

Teaching format, methods and strategies: Presentation with audience questions and answers

Speakers: Casey Marks, PhD,ISC2; Levi Boren, PhD, Pharmacy Technician Certification Board; Anthony Zara, PhD and Brenda Aubin,Pearson VUE

Certifying the Certifiers: Design, Development, and Implementation of the ICE-CCP Credential

Business of Certification; Level of Content: Intermediate

I.C.E. Exchange Program Description: In 2020, the Institute for Credentialing Excellence (I.C.E.) embarked on a process to develop the certified credentialing professional (ICE-CCP) certification program designed for individuals working in the credentialing industry. The program was built from the ground up, with several industry partners collaborating with I.C.E. to design and develop the key components of a credentialing program. This work culminated with the launch of the ICE-CCP pilot examination in November 2021. As members of our credentialing community may expect, this was a program without precedent. Both the subject matter expert volunteers who supported the test development process and the pilot test candidates represented a unique population of individuals: those who are currently employed in the credentialing business representing the variety of roles across governance, operations, and assessment development. In other words, no credentialing examination has ever been taken by such an informed group of test takers. Moderated by Linda Anguish, Director of Accreditation Services at I.C.E., this panel will include measurement professionals from the credentialing industry who contributed to the job analysis, item development, test publishing, test delivery, statistical analysis, standard setting, scoring, and reporting of the inaugural ICE-CCP examination. Hear about their challenges and reflections throughout the process, along with insights and recommendations for future program activities. Attendees are certain to be enlightened and entertained by the several unique and occasionally humorous observations and encounters. Attendees will have the opportunity to ask questions of the presenters and to learn more about the ICE-CCP if they are considering earning the credential themselves.

Teaching format, methods and strategies: A panel discussion among four psychometricians plus an I.C.E. staff member to serve as moderator and a resource to answer operational questions about the program.

Speakers:  Amin Saiar, PhD, Board of Pharmacy Specialties; Chris M. Beauchamp, PhD, Meazure Learning; Maria Incrocci, PhD, RPh, Scantron; Paul E. Jones, Ph D, Pearson VUE; Moderator: Linda Anguish, SPHR, Institute for Credentialing Excellence

Embedded Standard Setting for Credentialing: Eliminating the Traditional Standard Setting Workshop and Enhancing Validity

Credentialing Innovations; Level of Content: Advanced

I.C.E. Exchange Program Description: Embedded Standard Setting (ESS; Lewis & Cook, 2020) is currently employed by multiple K-12 educational state consortia and/or K-12 test publishers to support the coherence of their testing programs and fulfill their standard setting requirements. ESS transforms standard setting from a standalone workshop to an active part of the assessment development lifecycle that directly contributes to the assessment validity argument. Imagine—standard setting that eliminates the traditional standard setting workshop and improves test quality by maintaining a consistent focus on the evidentiary relationship between test items and their aligned claims and measurement targets throughout the test development lifecycle. In this session, presenters will describe the methods and results of an ESS pilot study. They will share results that indicate that ESS may be validly applied to support standard setting requirements in the certification industry. They will also will describe how ESS was adapted for the certification industry by using the Job Task Analysis and Minimally Competent Candidate descriptors as proxies for educational assessment performance level descriptors.

Teaching format, methods and strategies: Methods: Lecture and audience participation, where sample test item will be provided and the presenters will work with attendees to align items to a sample set of Minimally Qualified Candidate descriptors, discuss the consistency of the attendees’ alignments and empirical data, and demonstrate how ESS estimates cut scores based on the item alignments and empirical data.

Speakers:  Daniel Lewis, PhD, Creative Measurement Solutions LLC; Melanie Graw, MEd, National Restaurant Association

How fair is it to deliver your exams only in English?

Test Development and Administration; Level of Content: Introductory

I.C.E. Exchange Program Description: In the 21st century, accessibility includes language, but translating a test in all the different languages that people speak is not realistic. However, we don’t want pass/fail to be dependent on reading proficiency. So what are the possible solutions? One is to translate the test fully in some languages, for example many US programs should likely translate into Spanish. This needs to meet certain requirements. Another is to offer machine translation on the fly, to aid people with some knowledge of the base language that just needs supplementing. And a third is to reduce the reading proficiency threshold required to take the test (i.e. use simple language) This session will explore good practice in each of the three approaches. Translating into languages is covered by the NCCA Standards, and we’ll also share some good practice and technological approaches for all three.

Teaching format, methods and strategies: Concurrent session with audience polling and linkages to the NCCA Standards

Speakers:  John Kleeman, MA FBCS C.Eng CIPP/E, Questionmark; Steve Dept, MA, Capstan SA

Lessons learned from remote exam delivery 'The hard way': Best practices that came from these case studies

Test Development and Administration; Level of Content: Intermediate

I.C.E. Exchange Program Description: The last three years were a fast forward of assessment plans and many organizations were forced to adapt to a new modality. We learned a lot about how to successfully deliver remote exams. Unfortunately, many of our initial assumptions proved wrong as we did not realize what adjustments needed to be made in order to meet accreditation standards and deliver secure and fair exams. Explore the perils of remote assessment delivery from very experienced presenters. Three key case studies will be shared, including AOA board exams, NCA Law Exams, and a few other interesting cases. In the end, we made a lot of mistakes, made a lot of adjustments and are thrilled to discuss what we learned.

Teaching format, methods and strategies: Panel discussion.

Speakers:  Don Kassner, MonitorEDU; Steve Barkley, Paradigm Testing; Deborah Wolfe, P.Eng., National Committee on Accreditation and Law School Programs, Federation of Law Societies of Canada; Dan Hart, Certifying Board Services at American Osteopathic Association

Meeting Stakeholder Needs through Training and Credentialing

Business of Certification; Level of Content: Introductory

I.C.E. Exchange Program Description: Micro-credentials have been the topic of conversation for credentialing professionals for many years, and the industry is looking for concrete examples of how credentialing organizations are using them to expand their product portfolio. Attend this session to hear how a mission-driven organization used micro-credentials to meet the training needs of professionals in the field while also establishing the groundwork for a future professional certification program. In 2021, the Institute for Healthcare Advancement (IHA) launched a portfolio of seven micro-credentials for individuals working in the field of health literacy. The launch was the result of several years of planning and development to identify and understand the needs of the target audience, determine the appropriate credentialing product to meet those needs, and create a long-term strategy for the future. IHA engaged a team of credentialing experts to guide them through this journey using a framework based on in best practice and accreditation compliance. Leave understanding the challenges in the process and lessons learned through each project milestone.

Teaching format, methods and strategies: The session will be a panel format from three perspectives: (1) psychometrician; (2) staff and learning design; and (3) credentialing consultant

Speakers:  Cynthia Allen, ICE-CCP, SeaCrest; Rachel Roberts, MPH, CHES, Institute for Healthcare Advancement (IHA); Joy Matthews-López, PhD, JML Measurement and Testing Services; Shannon Carter, Ed., CAE, American College of Veterinary Internal Medicine (ACVIM)

Test Accommodations: Latest Trends from a Global Perspective

Business of Certification; Level of Content: Intermediate

I.C.E. Exchange Program Description: Many certification and licensure organizations have seen a significant rise in the volume of requests for test accommodations. Many (if not most) of these requests are accompanied by documentation from a psychologist, doctor, or other medical provider-- which provides a diagnosis—but not much more. This medical documentation tells us very little about the person’s actual functioning, their functional limitations, specific barriers they may encounter on our credentialing tests, or how the requested accommodations will enhance the person’s access to the test. In some countries, we are not even permitted to ASK what a person’s diagnosis is. This session will discuss the latest trends in test accommodations, from a global perspective, and will include practical strategies for implementation. Presenters will discuss the distinction between diagnosis and disability, the differences between the Social Model and the Medical Model of disability, and how these concepts can be implemented for reviewing test accommodations requests. The presenters will provide participants with strategies for retooling their accommodations practices to be more flexible and less reliant on medical documentation, and to emphasize functional limitations and barriers to access. Finally, the session will allow for significant audience participation and discussion.

Teaching format, methods and strategies: Traditional concurrent session.

Speakers:  John Hosterman, PhD, Paradigm Testing, LLC; Tamiko Hynes, MBA, CPA Canada; Lorna Martin, PhD, COMPASS Centre for Examination Development

Uncovering Bias in Your Test Content: Tools and Techniques for Identifying and Minimizing Potential Risk

Test Development and Administration; Level of Content: Intermediate

I.C.E. Exchange Program Description: With DE&I top of mind for many organizations, we need to recognize that bias can enter into any and all steps in the test development cycle. This session will focus on some ideas about how to identify potential sources of bias, how to measure and monitor for biases, and how to implement procedures to minimize bias potential.

Teaching format, methods and strategies: Presentation with audience discussion on techniques leveraged to identify and eliminate potential bias in content.

Speakers:  Jessica M. Anderson, ICE-CCP, Data Recognition Corporation

The Online Proctoring Gold Mine: blending data streams and synthesizing information to ensure validity and security of exams delivered online

Test Development and Administration

I.C.E. Exchange Program Description: During the shift to online proctoring, many programs were faced with a mountain of data that they never had convenient access to. Test centers & vendors historically stored that information and it was only requested when something went wrong. Summary incident reports are typically provided, but additional information is by request only. Investigations are often stifled by 3rd party actions that prevent timely response. Online is different, since everything is flagged near-real time, and direct access is provided to the program. Separately, platforms designed for online delivery support appropriate actions such as advanced search capability, watermarking, incident logs, and input tracking/warning. All of this DATA leaves a program drowning in information, but no guidance as far as WHAT to prioritize, and how to make the most out of a limited budget/resources. Review the common types of information available through online proctoring, and strategies for analyzing the data available to ensure that you're making the most of your time and money. 

Teaching format, methods and strategies: E-poster meet the author.

Speakers: Cicek Svensson, Caveon Test Security; Adel Lelo, Examity; Jill Finn, PMI-ACP,AAPC

The REBUILD Skills Act: an opportunity for certification organizations to offer federally funded scholarships and promote their brand

Business of Certification

I.C.E. Exchange Program Description: The REBUILD Skills Act is a game-changing piece of federal legislation that would permit most certification organizations to offer federally funded Career Rebuilding Scholarships to applicants who experienced job displacement during the pandemic, and to service members returning to civilian life and their spouses. The U.S. lost almost 9½ million jobs between February 2020 and February 2021, and economists forecast that some sectors of the economy may never return to pre-pandemic levels. After extended periods of unemployment or underemployment, many Americans need to earn private certifications or other credentials to be considered by employers. Congress recognized this in enacting this historic legislation. Certification organizations approved to participate can benefit from inclusion in a comprehensive federal database of qualifying programs. The authors of this poster were closely involved in drafting the provisions of this important legislation and supporting its passage. Learn the details about the REBUILD Skills Act, how Career Rebuilding scholarships will be implemented, how to navigate the approval process, and how your certification program can benefit from participation now and in the future. 

Teaching format, methods and strategies: E-poster meet the author.

Speakers:  Julia Judish, JD,Pillsbury Winthrop Shaw Pittman LLP; Jeff Evans, ASAE: The Center for Association Leadership

Continuing Quality Improvement’s Role in Social Media Strategy and Audience Engagement

Marketing and Communications; Level of Content: Intermediate

I.C.E. Exchange Program Description: Many in the credentialing space understand the principle of continuing quality improvement, or CQI, and its role in psychometrics, certification, and organization administration. However, it has a crucial role in social media that can sometimes be overlooked. In this presentation, members of the Board of Pharmacy Specialties Communications Team discuss: CQI in Social Media strategy, how to identify places for improvement, and how to establish routines to incorporate CQI into social strategy. Hear real-world examples of CQI in practice in a social strategy and accessible places for improvement regardless of where your organization is in its social strategy.

Teaching format, methods and strategies: Lightning Learning session

Speakers:  Kate McKenna and Alex Jean-Baptiste, PCM, Board of Pharmacy Specialties

Meet CNG - The Certification Networking Group

I.C.E. Exchange Program Description: CNG, the certification networking group, is a volunteer-driven, membership-based organization that offers invaluable credentialing industry connections, education, and resources for free. Come to the I.C.E. Booth to meet the CNG Board, and learn how to connect with this networking group.

CNG Board members:Pam Weber, CNG Board Chair; Katie Gottwaldt, Immediate Past CNG Board Chair; Terreline (Shellie) Sims, CNG Board Secretary; Amy McKee, CNG Board; Gayle Rosnick, CNG Board; Marya Ryan, CNG Board and Bill West, CNG Board

Chasing Relevance: How the Nation’s Emergency Medical Services Certification is Redefining Value

Marketing and Communications; Level of Content: Intermediate

I.C.E. Exchange Program Description: As the workforce and credentialing market continue to evolve, there is an increasing emphasis on the need to elevate the value of certification. Infrequently are we given the opportunity to witness an organization making the changes necessary. The National Registry of Emergency of Medical Technicians manages four certification programs and delivers over 175,000 examinations annually. At present, ‘The National Registry’ is redefining value. Through an overhaul of its certification exams, a deep dive into its brand and customer experience, and a deliberate engagement strategy toward the stakeholder primacy, the National Registry is breaking its status quo with high energy and integrity. Join members of National Registry’s leadership team for perspectives into the triumphs (and pitfalls) of organizational transformation. Specific attention will be placed on the National Registry’s “pivotal moment.” Participants will be given insight into lessons learned (thus far) and some strategic and tactical considerations which may guide decision making for their own organizations.

Teaching format, methods and strategies: Presentation with active engagement of the audience through questions and discussion of personal experiences

Speakers:  Alan Arguello, MBA, PMP, NRP, Meg Vavrinec, MBA, and Mihaiela R. Gugiu, PhD, National Registry of Emergency Medical Technicians

Client vs. Vendor Showdown: On the spot questions from client to vendor and vendor to client about the Analysis of Practice study. Who will survive?

Test Development and Administration; Level of Content: Introductory

I.C.E. Exchange Program Description: Have you ever wanted to see a showdown between a vendor and client? Hear the hard questions a client asks a vendor, or the questions a vendor wishes they could ask a client? Experience this at this session. The client (NCARB) will prepare questions for the vendor (Alpine Testing Solutions) regarding the large-scale Analysis of Practice study. The vendor will not know what questions the client will ask about the study. The vendor will also have questions prepared to ask the client. Any question is fair game and the other person will not hear the question until you do. As an example, the questions may include such topics as what was the most (or least) successful part of the Analysis of Practice study or what was the most painful part of working with us? The purpose of this session is not only to put the vendor in the hotseat (and maybe the client for a little bit), but to also inform the audience about the successes and failures of the different methods used to analyze the practice of architecture and how over 10,000 stakeholders were involved. Come to this session to watch faces turn red, but also to learn about an innovative approach to gaining insight into the who, what, when, why, where, and how of a professional practice.

Teaching format, methods and strategies: The first 10 minutes will be an introduction to the Analysis of Practice project and the relationship between the vendor and client. Then, a Q&A between the vendor and client will ensure for the rest of the session.

Speakers:  Amanda Wolkowitz, PhD, Alpine Testing Solutions, Inc.; Andy McIntyre, National Council of Architectural Registration Boards; Moderator: Laura Brooks, PMP, ICE-CCP, Alpine Testing Solutions

Getting Beyond “It Depends”: Psychometricians Debate Hot Topics in Credentialing - Part Three

Credentialing Innovations; Level of Content: Intermediate

I.C.E. Exchange Program Description: Attend this lively debate on hot topics and current challenges faced by credentialing programs. Join a panel of psychometricians as they spar over topics such as whether a firewall between assessment and education is truly needed, whether different groups should have different passing standards, whether the typical approach to testing for item bias is sufficient, and whether common credentialing practices yield sufficient validity evidence. After presenting positions and rebuttals on each issue, the panel will entertain questions from the audience and discuss their own opinions on each topic.

Teaching format, methods and strategies: Debate

Speakers:Amin Saiar, PhD, Board of Pharmacy Specialties; Daniel H. Breidenbach, PhD, ICE-CCP, PSI; Tim Muckle, PhD, NBCRNA; Sarah Carroll, PhD, Board for Certification in Occupational Therapy; Andrew C. Dwyer, PhD, ICE-CCP, American Board of Pediatrics:

Practice Makes Perfect: How to Prepare a Successful Re-accreditation Application

Standards and Accreditation; Level of Content: Intermediate

I.C.E. Exchange Program Description: Earning NCCA accreditation is no easy task, and once you do, that’s just the beginning! The next horizon is just five years away…reaccreditation. The stewards of 11 currently NCCA-accredited high stakes certification programs share lessons learned during the development of 19 NCCA accreditation and reaccreditation applications. Speakers will discuss how staying focused on accreditation after you earn it will drive continuous improvement in your program, enhance fairness and equity, strengthen your governance board, and make re-accreditation less stressful. Learn tips on how to weave accreditation into your day-to-day operations, and how to maintain a culture of quality among your front-line staff. Understand the importance of self-study and the I.C.E Annual Report, and hear methodologies to identify problems early, analyze solutions, make decisions, and record lessons-learned to maintain forward progress in a rapidly changing environment. Learn how to efficiently generate justifications and capture artifacts that demonstrate compliance with NCCA Standards and highlight the Assessment Development, Psychometric, Governance, Data Management, Test Operations, and Maintenance Essential Elements that NCCA commissioners emphasize during the reaccreditation process. Bring your questions and share your perspectives during an interactive presentation of how to achieve reaccreditation with minimal inquiries!

Teaching format, methods and strategies: Traditional concurrent session

Speakers: David A. Renberg, MS, US Government - National Geospatial Intelligence Agency; Chrissy M. Noel, MS Industrial/Organizational Psychology; ICE Credentialing Specialist,GEOINT Professional Certification (NGA)

Snakes in the Grass: Protecting Your Program from Remote Access Proxies

Security, Records, and Data Management; Level of Content: Intermediate

I.C.E. Exchange Program Description: Where once testing programs would confront impersonators and imposters in-person, the mass migration to online remote testing has forced programs to combat these fraudsters through the ether. Proxies are now taking tests by remotely accessing test takers’ computers. This form of cheating is increasingly common and incredibly hard to detect. Accessing a test taker’s computer is now easy (and profitable). Not only can scammers advertise their services on social media, but they can even prospect for new business. How can a program protect itself from these snakes in the grass? By working together, test professionals can unite and work collaboratively to identify known proxy rings and render them impotent. This session will share key lessons learned in confronting these proxies and their impact on test results. Specifically addressing new methods, both simple and sophisticated, that can be employed to subvert remote access proxies. Leave this session with an understanding of the impacts of a rapidly-growing challenge to test score validity, and proven tools to confront the problem head on. Don’t miss this opportunity to get ahead of the fraudsters and thwart upcoming attacks to your program.

Teaching format, methods and strategies: Traditional concurrent session

Speakers: Steve Addicott, Caveon; Jon Jensen, Examity; Pete Van Dyke, Amazon Web Services (AWS)

Success factors in developing and implementing competency models

Test Development and Administration;>Level of Content: Intermediate

I.C.E. Exchange Program Description: Competency modeling is an alternative or complement to practice analysis as a means of describing the requirements for practice in a profession. Developing a competency model represents an investment for a credentialing organization. The focus of this session is on how to maximize the organization’s return on that investment. Competency models can inform a range of education, examination, and professional development activities. Identifying and planning for these uses up front, combined with impeccable execution, and strong stakeholder communications and involvement are key ingredients for successful implementation and acceptance. Attendees at this session will learn about success factors in planning for, developing, and using competency models for a range of applications not only in licensure and certification programs but in developing a competent workforce. Presenters are experienced in developing and implementing competency models for licensure and voluntary certification organizations and will share their recommendations and lessons learned for maximizing the utility of competency modeling for professions.

Teaching format, methods and strategies: Traditional concurrent session.

Speakers:Patricia Muenzen, PhD, ACT; Adele Foerster, MSN, RN, CPNP-PC/AC, ICE-CCP, Pediatric Nursing Certification Board; Jan Robinson, College of Veterinarians of Ontario

Thinking outside the box: Leveraging microcredentials across multiple credentialing programs.

Beyond Traditional Credentials; Level of Content: Introductory

I.C.E. Exchange Program Description: What are microcredentials? How might they apply or be an option for credentialing in various disciplines? Why are we even talking about this and what are key considerations and benefits of microcredentials that make this discussion critical and timely? A panel of credentialing program leaders discuss and brainstorm potential applications of micro-credentials within and across nursing specialties as well as applications across other disciplines. Hear key benefits as well as challenges related to microcredentialing, generate ideas, and review future opportunities and how microcredentials could be leveraged to continue support the mission of credentialing organizations.

Teaching format, methods and strategies.: Panel Discussion, with key questions posed to the panelists and then additional questions for the audience.

Speakers: Isabelle Gonthier, PhD, ICE-CCP, PSI Services; Tony Ellis, CAE, ICE-CCP, Oncology Nursing Certification Corporation; Terri Hinkley, EdD, MBA, BScN, RN, CAE, Medical-Surgical Nursing Certification Board; Janie Schumaker, MBA, RN, CEN, CENP, CPHQ, FABC, Board of Certification for Emergency Nursing

Best Practices for Real-World Conversations About Statistics and Measurement

Test Development and Administration; Level of Content: Intermediate

I.C.E. Exchange Program Description: In this game-show style panel session, statistics and measurement experts from well-known credentialing program will respond to psychometric-related questions that are commonly asked by business leaders, stakeholders, and others. In doing so, they will explain how to balance the science of I-O psychology, psychometrics, and related fields with the realities of practice, sharing from their own experiences to illustrate a framework and best practices for formulating responses that build understanding and drive desired outcomes. The goal of this session is to provide a framework and best practices for I-O psychologists and other assessment experts to leverage when addressing statistics and measurement questions from business leaders and other stakeholders associated with high-stakes testing programs.

Teaching format, methods and strategies: Game Show: During a one-hour session, the chair, serving as the game show host, will ask a statistics and measurement question followed by the most common answer (i.e., textbook response). 

Speakers: Bethany Bynum, PhD,  and Kimberly Adams, PhD, Human Resources Research Organization; James P. Henderson, Credentialing Examination Consulting, LLC; Tara Myers, PhD, American Nurses Credentialing Center (ANCC); Liberty Munson, PhD, Microsoft

Facilitating Item Development: Reducing SME Time Demands, Reusing Distractors, and Recycling and Distilling Item Components

Credentialing Innovations; Level of Content: Advanced

I.C.E. Exchange Program Description: SME time is increasingly valuable as testing program content requirements become more complex. In this session, explore natural language processing and item bank text mining strategies to facilitate item development and conserve SME time. The presenters will describe and discuss thei methods they sued, results, and SME perspectives.

Teaching format, methods and strategies: Presentation, Case Study, Q&A

Speakers: Tara McNaughton, PhD, Jean D’Angelo, BA, Nicholas Williams, MEd, and Eden Racket, BA, American Osteopathic Association

Getting the Best from Your Credentialing Board Public Member

Business of Certification; Level of Content: Intermediate

I.C.E. Exchange Program Description: Attend this session to discuss challenges and identify solutions for identifying, recruiting, and engaging public members. This session is facilitated by members of the I.C.E. Public Member Committee, and the solutions generated will be used to create an I.C.E. resource.

Teaching Format, Methods, and Strategies: A facilitated round-table discussion in which attendees will break into small groups to answer questions, discuss issues, find solutions to problems, and share their findings with other participants.

Speakers: Nancy T. Ahluwalia, MS, MBA, NCCA; Beverly Black, MHSA, CAE, NCCPA Board; Karen Plaus, PhD, CRNA, FAAN, FAANA, I.C.E.; Glenna Tinney, MSW, ACSW, DCSW, Captain, U.S. Navy (Ret.), National Board for Respiratory Care; Sheryl Traficano, MBA, CAE, Certification Board for Diabetes Care and Education

Preparing for Pivotal Moments in the Lifecycle of Your Certification Program

Business of Certification; Level of Content: Intermediate

I.C.E. Exchange Program Description: Inevitably, most certifiers will face pivotal moments in their credentialing programs - moments when they must make their most difficult decisions, such as whether to sunset a faltering credential or to redesign a highly visible, flagship credential. One of the most challenging roles of a certification professional is to effectively lead the organization through the process. Foreseeing that it would soon be confronted with two pivotal moments, the Institute for Internal Auditors took action to prepare proactively for these circumstances. Recognizing that its advanced credential was underperforming, staff created a task force to uncover factors contributing to the credential's performance and evaluate whether alternative products might be more desirable to the market. Noting that professional and credentialing trends and market demographics could impact participation in IIA's "gold standard" certification in the future, senior staff launched a strategic initiative to explore these changes and determine whether and how to "refresh" the program to better align it with market needs and trends. Using a disciplined process, IIA staff effectively guided volunteer leaders through exploring and reflecting on the meaning of the pivotal moment for each credential and then deciding the best course for the future. This carefully developed process included conducting targeted market research, elevating the voice of customers in the decision-making process, prioritizing data over opinions and past practices, and anticipating and addressing stakeholders' hesitancy to change. Join us to analyze and discuss these two case studies and to gain additional insights from other participants on how to most effectively lead and support staff and volunteer leaders in pivotal moments.

Teaching format, methods and strategies: Case study presentation with audience discussion

Speakers: Lenora Knapp, PhD, Knapp and Associates, International; Lily Bi, CIA, CRMA, QIAL, CISA,The Institute for Internal Auditors; Sandra Bauman, PhD, Bauman Research & Consulting, LLC

Taking an Exam: The Candidate Experience

Business of Certification;>Level of Content: Intermediate

I.C.E. Exchange Program Description: Taking an exam is stressful. Test anxiety is real and can cause candidates to fail an exam over and over based on a job they are already performing. Do we care? Should we care? We say yes. Candidates invest time and money preparing and sitting for exams. While, credentialing bodies focus on the security and integrity of the exam, evaluating the candidate's experience is often an afterthought, if even considered at all. If you do care, what can you do to make the experience less stressful for the candidate? Please join us as we discuss candidate perspectives and expectations and the variety of ways that you can attract your desired candidate pool.

Teaching format, methods and strategies: Panel discussion with audience engagement based on scenario activities.

Speakers: Nicholas Soto, CAE, Construction Management Association of America; Johnna Gueorguieva, PhD, Dental Assisting National Board, Inc.; Bill West

Three Case Studies of Innovative Certifications

Beyond Traditional Credentials; Level of Content: Advanced

I.C.E. Exchange Program Description: Attend this session to hear from representatives from three organizations: - the International Society for Performance Improvement (ISPI) that has a 20 year old portfolio-based certification whose certificants work across all industries in 60 countries, the National Association for Roofing Contractors (NRCA), that rolled out its first of 17 performance-based certifications in 2019 each with a hands-on assessment, and the Institute for Performance Improvement (TIFPI) that offers a project-based certification for facilitators of multi-sector initiatives to advance the development of people in careers where there is significant skill shortage (airline industry for example). The panelists will answer four questions: What was the problem you were trying to solve?What is different about your certification compared to traditional test-based credentials?How does your model address your business or societal need? What have you learned that others might benefit from knowing?

Teaching format, methods and strategies: Panel discussion with moderated Q&A.

Speakers:Nancy C. Burns, PhD., CPT, PMP, CPIF, International Society for Performance Improvement; Deb Page, CPT, CPIF, HPLJ, Institute for Performance Improvement; Jeanne Schehl, EdD, National Roofing Contractors Association; Moderator: Judith A. Hale, Center for International Credentials

Applying for Accreditation under the NCCA 2021 Standards (NCCA Session): A session with NCCA Commissioners 

Standards and Accreditation; Level of Content: Beginner

I.C.E. Exchange Program Description: As part of its continuous process of quality improvement, the NCCA initiated a review and update of the Standards beginning in August 2019. The intent of the revisions was to clarify, reorganize/increase efficiency, and address challenges that have arisen in applying the Standards over the past four years. Through this process, the 2021 NCCA Standards clarifies the requirements that programs have been addressing through inquiry and compliance reporting processes over the last several years. Attend this session to improve your understanding of the 2021 NCCA Standards, the practical implications of the revisions, and how to submit a well-documented and supported accreditation package. This session will be especially helpful for programs who are preparing reaccreditation applications.

Teaching format, methods and strategies: Traditional concurrent session.

Speakers: Kevin Bradley, PhD, Human Resources Research Organization (HumRRO); Natalya Mytareva, MA, CoreCHI™, Certification Commission for Healthcare Interpreters; Michele Warholic

Delivering Fair and Defensible Performance Based Exams

I.C.E. Exchange Program Description: Good performance based exam design can often be a delicate balancing act. How do you successfully deliver a performance scenario to your candidates without inadvertently guiding them to the correct answers? How do you keep the integrity of the testing environment? How do you maintain defensibility against candidates who are convinced they know the correct answers even though the exam data says otherwise? Learn from the experts as we talk through how we've addressed these challenges in our own IT certification programs and have implemented changes which put candidates on the road to success in a performance based exam environment.

Teaching format, methods and strategies: A combination of a presentation and panel discussion, including interaction with the audience to share their own experiences with performance based exams.

Speakers: Pat Hughes, Internet Testing Systems; Jefferson Hansen, Certiport; Liberty Munson, PhD, Microsoft; David Elfassy, Netlogon

Diversity, Equity, and Inclusion: Key Considerations for Policy and Delivery Changes

Credentialing Innovations; Level of Content: Intermediate

I.C.E. Exchange Program Description: One positive outcome of the pandemic has been a renewed focus on equity, diversity, and inclusion in everything we do. And testing and certification are no exception. Credentialing organizations, regulators, and hiring practices are increasingly focused on ensuring assessment fairness across a broad population of test-takers. The organizations responsible for developing and delivering these assessments have been diligently working on attracting a diverse population of test-takers, improving accessibility through hybrid delivery models, and implementing designs that minimize bias. In this session, panelists will discuss challenges and opportunities related to diversity, equity, and inclusion from three unique perspectives, including a large credentialing organization in food safety, a certifying body in the finance industry , and a nursing regulator. The panelists will explore key diversity and inclusion considerations as they build and promote their programs and partners, ensuring equity for all test-takers. They will discuss initiatives they have implemented, including hybrid delivery models, and share positive outcomes and lessons learned.

Teaching format, methods and strategies: A panel presentation in which the panel will receive 5 questions to answer.

Speakers: Larry Lynch, FASAE, CAE, National Restaurant Association; Elizabeth Stewart, CFP Board; Jennifer Breton, College of Licensed Practical Nurses of Manitoba; Moderator: Ashley Norris, PhD, Meazure Learning

Increasing the Efficiency and Security of Remotely Delivered Tests

Test Development and Administration; Level of Content: Intermediate

I.C.E. Exchange Program Description: Whether we wanted it to or not, remotely delivered tests have become a reality. This change was so rapid that most programs had no choice but to convert their traditional tests to remote delivery and to risk the exponentially increased item exposure that comes with it. The obvious answer is to create more new items and replace test forms more often. Unfortunately, this means significant increases in test development labor and test publication costs. A better approach is to adopt newer item types that produce significantly more measurement opportunities per item and are much easier to clone than traditional multiple-choice items. Not only can measurement opportunities be increased and produced more rapidly, but the items are themselves less vulnerable to the effects of exposure and can test for higher-level professional skills. This presentation looks at how enhanced case management items can provide two to three times as many measurement opportunities as the same length of traditional multiple-choice items why they are easier to clone and more secure, all of which makes them ideal for remote test delivery.

Teaching format, methods and strategies: Traditional concurrent session with audience discuss of issues presented.

Speaker: Clarence L. Chaffee, The Caviart Group, LLC

Maximizing Impact of Research & Development Investments

I.C.E. Exchange Program Description: Getting maximum benefit from investments in research & development activities is an obviously important but all too often unrealized goal. Such activities might include routine requirements such as occupational analysis as well as R&D to support changes to policy, assessment methods, administration modes and protocols, and so forth. Join this discussion of barriers to conducting impactful R&D, strategies for maximizing the value of R&D activities, and specific examples of barriers and mitigation strategies from both the presenters and audience members. Session will be equally divided between presentation and audience discussion. Bring your questions, ideas, and curiosity to this session.

Teaching format, methods and strategies.: Presentation, facilitated audience discussion, and a wrap-up to reinforce key take-away points informed by presenter and audience input.

Speakers: Deirdre J. Knapp, PhD, Human Resources Research Organization (HumRRO); Tim Muckle, PhD, NBCRNA

Privacy and Data Protection: Compliance Annoyance or Candidate Safeguard? Guidance on Effective Privacy within a Certification/Credentialing Program

I.C.E. Exchange Program Description: This session by two experts in assessment and privacy sets out the key steps for a certification or credentialing program to ensure that it respects the privacy of its candidates and is likely to meet data protection regulatory requirements. Privacy rules and regulations are in flux worldwide. Europe is stepping up its enforcement of the GDPR, and there are several newly enacted US State and international privacy laws with even more proposed laws circulating through legislative channels. This session will give an overview of the privacy requirements faced by credentialing and certification bodies, focusing primarily on the US and Europe. Some credentialing specialists see privacy and data protection as a compliance burden. However, this session will suggest ways in which programs can comply with multiple laws and regulations and actually show respect for candidate privacy by ensuring personal data is safeguarded and only used for the purpose of an assessment, which can also be a candidate safeguard and a marketing benefit. The session will also suggest that by preparing properly to meet privacy needs, you will save time and effort in the long term.

Teaching format, methods and strategies: Presentation with pre- and post-presentation audience participation

Speakers: John Kleeman, MA FBCS C.Eng CIPP/E, Questionmark; Allison Mulford, ESQ, MBA, CIPP/E, Caveon, LLC