#test validity

Reliability - the test measures the specific construct consistently and accurately. It discriminates between low scorers and high scorers. You should expect to see similar results from the same test taker with a different version of the test, and across different scorers.

Validity - encapsulates reliability, as well as adequate and appropriate interpretation of scores. Does the test actually measure the specific construct? And is it appropriate to base decision making on evidence of this construct?

Adequate content and skills coverage - The target learning outcomes are covered. The test taker must recognise the correct answer, produce the correct answer or perform in a dynamic context as required.

Practicality - design, administration and scoring are practical.

Impact - the real life consequences are fair. The test will enable decision makers to make good decisions.

Issues/threats to validity

Construct under-representation - the most relevant aspects of the construct are not tested.

One of the ones we looked at when I was in school was an IQ test that had two drawings- one was a man in a suit and one was a man in jeans and a t-shirt. The question asked kids to circle what the man would wear on the weekends. The 'answer' was supposed to be the man in the jeans and t-shirt, but inner-city black children almost always chose the man in the suit. Why? Because most of the men in their lives were blue-collar workers who wore jeans (dungarees, more specifically) and t-shirts to work during the week, but they dressed in their best for Sunday church. Most of these kids had only seen men in suits during the weekend. The test was culturallyn and socio-economically invalid.

As the usage of unproctored Internet testing (UIT) increases in selection settings, concerns about the validity of such practices must be addressed. While recent examinations of the issue have focused primarily on the practicality, ethics, and potential legal ramifications of UIT, this paper provides an examination of the criterion-related validity of unproctored assessments. Using a database of validity evidence, we examine the predictive validity of several noncognitive assessments (i.e., personality and biodata) with respect to a number of subjective and objective job performance metrics. Results generally indicate that assessments administered in proctored and unproctored settings have similar validities. Limitations of this research and implications for practice are discussed.

View full article here: http://www.ingentaconnect.com/content/bpl/ijsa/2011/00000019/00000001/art00001