Standard Setting
Facts
The standard for an exam (commonly referred to as the cut score) is the scoring threshold that an examinee must meet or exceed to pass the exam. It is important that a cut score be set appropriately for both the difficulty of the items and for the ability level being tested. To ensure fairness, the cut score must not be arbitrarily defined. CMS can help you determine a fair and legally defensible cut score for each of your exams.
How is Standard Setting performed?
There are different industry-accepted methodologies for setting the cut score. Typically, CMS facilitates multiple sessions — either in a face-to-face setting or via an online, web meeting setting — with a group of subject-matter experts (SMEs) to discuss descriptions of the barely-competent candidate for a specific exam and the likelihood of that group of candidates to successfully responding to each item in the test bank. In one methodology, SMEs may be asked to respond to each test item and then estimate a difficulty level of each item. Using a different methodology, SMEs may be asked to estimate a point on a continuum of item beta statistics at which more candidates will begin to answer items incorrectly than the number of candidates answering items correctly. CMS ensures that the appropriate standard setting methodology is used to determine the exam cut score.
What outcomes should you expect from a Standard Setting service?
Outcomes from a Standard Setting service normally include:
- Documentation that an industry-accepted standard setting methodology was used to determine the exam cut score
- Assurance that valid, reliable decisions and judgments can be made on the outcomes of candidate testing on the basis that an appropriate cut score methodology was employed