Evaluating the quality of a behavioral assessment instrument : a treatment validity study

The purpose of this study was to evaluate the treatment validity of the State-Event Classroom Observation System. Treatment validity is one application of the overall notion of validity which is concerned with determining the appropriateness of interpretations made from test scores. No test is valid for all purposes. The degree of validity depends on a particular use; therefore, one must always ask, validity for what? In treatment validity, the question is one of the relationship between assessment and treatment outcome. Treatment validity is established through evidence that an assessment instrument contributes to the effectiveness of treatment. For example, if an instrument identifies appropriate target behaviors for change, these behaviors can serve as pre-intervention measures against which behavior change can be compared. Similarly, if an instrument is sensitive to changes in target behaviors, it can be used to help determine when modifications in recommended intervention procedures are warranted.

Support for the treatment validity of SECOS was provided through evidence that the code is sensitive to both known behavioral differences distinguishing ADHD children from their peers, and the effects of stimulant medications on these behaviors as documented in numerous studies. Eighteen pairs of hyperactive and normally active children were observed in their regular classrooms both pre- (i.e., at referral) and post-medication of the hyperactive children. At referral, SECOS data indicated that ADHD children were more frequently off-task and exhibited more total disruptive behaviors and more variable behavior than comparison children. Following medication, previously documented enhancements in the behavioral functioning of ADHD children (i.e., increased attention, decreased disruptive behavior and behavior variability) were evident on SECOS. The findings in this study were further substantiated by analogous changes in children's daily academic work. Results are discussed in terms of their implications for identification and treatment of hyperactive children.