Constructing and Critiquing Level Two
Evaluation and Certification Systems

Sharon Shrock, PhD and William Coscarelli, PhD

Many instructors, trainers, performance technologists, and human resources professionals have had little experience with test development and possess few skills with which to address management requests for data verifying what employees can do or management's requests to create certification tests. Tests that truly measure what competencies people can do are called criterion-referenced tests (CRTs). 

Creating this type of test for Level Two assessment or certification (either internal or external) is often difficult because CRTs do not look like the school-based or nationally distributed norm-referenced tests we have all experienced when teachers graded on the curve or when we took the ACT or SAT. 

Simply being able to write good test items will not be adequate in today's competitive and litigious environment. Specific knowledge of the elements of building a defensible and effective assessment system; for example, documentation, setting mastery scores, and test validation is required. This workshop provides an overview of the CRT development process and includes specific exercises and examples to highlight critical skills and emphasize the legal issues that must be addressed. 

When you leave the workshop you will have the basic tools to critique currently existing tests in your organization as well as turnkey skills for creating defensible Level Two assessments or certification tests.

Objectives

Participants will be able to:

• Determine when to use criterion-referenced tests and when to use norm-referenced tests.
• Define the 14 steps to follow to create a professionally and legally defensible test for Level Two assessment or certification.
• Create technically correct and job-related multiple-choice items.
• Develop objective tests to measure performance while balancing the legal rights of test takers with special needs.
• Create reliable performance tests that measure skills while demonstrating "job relatedness" in the legal context.
• Determine test length. 
• Determine defensible mastery levels (cut scores).
• Describe defensible procedures for establishing reliability and validity.
• Identify the critical steps to developing legally supportable certification tools and test instruments.
• Develop a specific plan for documenting work to minimize legal risks.

Intended Audience

Anyone who will have to create or critique a criterion-referenced assessment or anyone is contemplating its implementation. No statistical skills assumed.

  << top >>