Analyzing test unity

This week we focused on test unity, how useful, practical and efficient is the test. Analyzing test unity, the decision theory comes into play, Statistical decision theory focuses on the investigation of decision making when uncertainty can be reduced by information acquired through experimentation. (Parmigiani,2010) with decision theory we can follow the guidelines to create the optimal cut-off score within tests, promoting a unified test. The cutoff score is defined as a (usually numerical) reference point derived as a result of a judgment and used to divide a set of data into two or more classifications, with some action to be taken or some inference to be made on the basis of these classifications. ( Cohen. Swerdlik,2018)

Classical Test Theory

The Classic test theory or CTT is widely known and it is the aspects of a total test score made up of multiple items, this theory is based on the test takers performance and correctly marked answers. The CTT boast several advantages like the simplicity of usage, it does not require a complex understanding, large sample, or vast ability to interpret. The CTT collectively looks at groups of testers and evaluates their success rates.  Many states that limitations are found in the circle dependency of this approach with respect to the statistic and samples.

Item Response Theory

This approach focusses mainly on difficulty level, instead of correct answers on a test. Here instead of correct answers setting the parameter we see a certain level of understanding or difficulty becoming the benchmark. IRT is more theory grounded and models the probabilistic distribution of examinees’ success at the item level. As its name indicates, IRT primarily focuses on the item-level information in contrast to the CTT’s primary focus on test-level information. (Fan,1998) The advantages of the IRT is that it allows for a more comprehensive description and detailed purpose of the test. Disadvantages of IRT model is with sample sizes, the large the sample size the more accurate the evaluation, and complexity.

Between the two CTT and IRT, I would have to say that IRT is the preferred theory due to the diverse and complex ability to test fairness. Rather than an overarching formula being applied to vastly different populations, IRT allows test developers to tailor tests around the traits of the individuals being tested. IRT tests also undergo a rigorous process of developing item banks, reviews, and tests before the finished test is implemented ( Cohgen,2018) Lastly, the diverse shaping and analyzing tests help to limit test bias.

References

Cohen, R. J., & Swerdlik, M. E. (2018). Psychological testing and assessment: An introduction to tests and measurement (9th ed.). New York, NY: McGraw-Hill.

Fan, X. (1998). Item response theory and classical test theory: an empirical comparison. Educational and Psychological Measurement .

Parmigiani, G. (2001). Learn more about Decision Theory. International Encyclopedia of the Social & Behavioral Sciences.