The 10 best things not to do when seeking accreditation of a US Government certification program

Track: Test Development and Administration

Session Number: 4063
Date: Wed, Oct 25th, 2017
Time: 1:30 PM - 2:15 PM


A humorous examination of serious issues related to developing a legally-defensible NCCA compliant certification program for the US government.  This fast paced discussion is based recent real-world challenges encountered by a US Government certification team and will be facilitated by the Director of 11 US Government certification programs, their accreditation team lead, and their lead psychometrician. Presentation outline:

  1. Don’t hand out certifications like candy
    1. Programs rely on SMEs to develop test questions
    2. Exposure to the seed bank may disqualify SMEs from exam participation
    3. Limit exposure and grant certifications only after evaluating knowledge/skills
  2. Don’t assume US Government/organizational goals neatly align with NCCA Standards
    1. Existing Work Role Studies/JTAs may not be sufficient for high-stake certifications
    2. Establish compliant standard setting panels and train your members
    3. Credentials must serve all stakeholder populations (military, government agency, industry)
  3. Don’t underestimate your psychometrician
    1. If unable to use one method (publish classified data) use another (sampling tables)
    2. Psychometricians understand testing statistics better than most of us
    3. Item level and exam level performance matters
  4. Don’t let governmental bureaucracy and criticism slow you down
    1. Not all candidate communications will be supportive
    2. NCCA inquiries are not affronts – they are a method to confirm program compliance
    3. The Governance Council should scrutinize
  5. Don’t let creeps weird out your exam administrators/proctors
    1. Not all candidates follow the rules
    2. Proctor guides support consistent test environments
    3. If it hasn’t happened, it will soon
  6. Don’t let flash mobs steal your exams
    1. Exam security is paramount
    2. Use best practices and invest in preventative measures
    3. Ongoing analysis is worth the cost
  7. Don’t tell the senior practitioner he failed when he didn’t take the test
    1. Mistakes will probably occur
    2. Address errors immediately
    3. Develop sound processes and train your people
  8. Don’t let candidate records become the enemy
    1. Candidate management systems don’t care if you input bad data
    2. The numbers may not want to support your agenda
    3. Annotations may be just as important as the data itself
  9. Don’t give up on your accreditation application
    1. Inquiries are opportunities to demonstrate you nailed the standard
    2. Programs evolve: be aware of community trends, innovate, improve
    3. NCCA wants you to succeed – even if you are different (USG Intelligence Agency)
  10. Don’t lose your sense of humor
    1. As long as you properly address the problem, you can laugh about it later
    2. It may not be as funny when presenting it to the board
    3. Certification programs are not a game but can still be entertaining
Session Type: Roundtable

Session Type: Roundtable