A Reliability and Validation Study of an Administrative Effectiveness Survey for Deans at Institutions of Higher Education

The purpose of this study was to revise and to establish the reliability and validity of the Administrative Effectiveness Survey and to field test the survey on the performance of academic deans at selected institution of higher education. Content validity for the instrument was, in part, established by asking two expert panels to review the instrument and make comments. Revisions to the initial instrument were made based on the suggestions by those experts. As a further test of content validity, factor analysis was performed on the data from the national study.

The instrument was administered to 677 faculty members at four colleges and universities across the country and 179 faculty members completed the instrument for a response rate of 26%. The instrument contained five different factor areas, planning and goal setting, leadership style and skill, resource stewardship, values and standards, academic stewardship, and a question about the overall performance of the academic dean. Results from the factor analysis revealed one primary factor, leadership, and two much smaller and less significant factors, external academics and mission, standards, and resources. The data were also subjected to reliability testing. A reliability coefficient of .983 was produced by the Cronbach Alpha.

Low response rates by the national expert panel, institutional participation, and respondents to the instrument resulted in less than desired outcomes of the study. The statistical analyses, while meeting minimal qualifications for conducting them, were not strong indicators of validity and reliability. Thus, the Administrative Effectiveness Survey cannot be deemed valid and reliable at this time without further research.