Statistical Validation and Item Selection
Facts
For new items, it is advisable to conduct a beta test to gather empirical data on each test item. The data is then analyzed statistically to determine if the test items and each of their answer options are performing satisfactorily. For example, the higher a person’s ability level, the higher the probability should be of their correctly answering any given test item. After conducting the statistical analysis, we make recommendations as to which items should be used to make pass/fail decisions. The same type of analysis is also done on operational exams to verify that items are properly discriminating and that the forms are balanced.
How is Statistical Validation and Item Selection Performed?
Statistical validation is determined using industry accepted statistical procedures such as the Classical Statistics Report and the Item Options Report. These reports provide different statistical information on item and item option performance. Depending upon client needs, other types of statistical analysis processes can also be employed. After the analysis is complete, psychometricians confer with the test owner to explain the report outcomes and to make recommendations for items to use in live testing.
What outcomes should you expect from a Statistical Validation and Item Selection service?
Outcomes from a Statistical Validation and Item Selection service normally include:
- Statistical data on each test item that validates item readiness for testing
- Identification of specific test items that can be used on test forms for live testing