An investigation of reaction to psychological measurement

The purpose of this study was to determine if a measurable relationship exists between subjective reactions to a test/testing situation and the reliability of the testing instrument. A review of the literature revealed a paucity of research dealing with the influence of subjective reactions on the psychometric properties of tests. The major emphasis in the literature concerned research dealing with test-anxiety. This literature was summarized in terms of the negative impact on performance of a number of variables associated with test-anxiety.

A questionnaire type instrument was presented to a sample (N = 233) of undergraduate university students using two instructional conditions. The instructional conditions consisted of a high reaction set of instructions designed to elicit more negative reactions by having the administrator of the test act in a rude and aversive manner and a low reaction condition which was intended to elicit a more positive participant reaction. The questionnaire consisted of four sections: demographic items, subscales from the Edwards Personal Preference Schedule (EPPS), a Reactions Questionnaire, and the State- Trait Anxiety Inventory (STAI). The complete questionnaire contained a total of 189 items. The order of presentation of the four sections of the questionnaire was held constant through the study. The EPPS subscales used in this study were the order, affiliation, endurance, and aggression scales. The Reactions Questionnaire was a modified version of the one developed by Friedenberg (1978) and was intended as a measure of subjective reactions to the EPPS in such areas as participant reaction to item content and the context of administration.

A factor analysis performed on the responses to the Reactions Questionnaire revealed five orthogonal reaction factors that accounted for 76.7 percent of the common variance. Factor scores were computed for each individual on each of the five reaction factors. These factor scores were then rank-ordered and were used as a basis for splitting the sample into separate "high" and "low" subgroups after dichotomizing and trichotomizing the sample.

Three hypotheses were developed and tested. The first hypothesis was that individuals experiencing more negative reactions (i.e., those in the "high reactions" subgroups) to the EPPS would show lower EPPS reliabilities than individuals experiencing more positive reactions to the EPPs (i.e., those in the "low reactions" subgroups). This hypothesis was not supported. The second hypothesis was that individuals receiving the high reaction instructions would show lower EPPS reliabilities when compared with individuals receiving the low reaction instructions. No support was found for this hypothesis. The third hypothesis that subjective reactions would be significantly related to state-anxiety was partially supported. The findings are discussed in terms of their implications for research in related areas as well as for further research on the relationship between subjective reactions and reliability. Reasons for the failure to achieve significant results are identified and possible ways of overcoming these difficulties are discussed. Areas for future research on this topic are identified.