A comparative study of needs assessment methodologies

This study tested the validity of the survey of knowledgeables needs assessment technique. Criterion-validation techniques were employed in this analysis using the target population needs assessment technique to measure validity.

Two surveys were conducted in the 15 counties located in the ETDD. The first, a face-to-face interview of 1,003 elderly residing in 732 households in rural areas of these counties was conducted in winter 1976-1977. This survey explored the social service needs of elderly living in rural areas. The second, conducted during the first six months of 1981, consisted of 75 telephone interviews with a purposeful sample of knowledgeables, representing six occupational groups, working in the study area. This survey assessed the needs of rural elderly by obtaining estimates of service need from knowledgeables employed in human service professions.

When responses to the service need questions in the target population interview were tabulated, percent of need for services varied by service and county. A pattern emerged where most counties had similar relative need ranks when compared to other counties. When service needs were ranked, there was some consistency of rank placement. The need for transportation was most often cited as the greatest need with seven counties citing it with a rank of eight. On the other end of the spectrum, companion services was ranked as the lowest service need with 12 counties giving it a rank of one.

Great discrepancies between knowledgeables' need estimates were shown when responses to the service need questions were tabulated. The standard deviations associated with the mean estimates were consistently large. Sixty-six percent of these standard deviations were above 19. The lack of a mode in more than 50 percent of the service need groups further illustrated this inconsistency. The large standard deviations coupled with the mode data indicated that knowledgeables were unable to reliably estimate the service needs of rural elderly.

Comparative analysis techniques including Spearman rank order correlation and the calculation of mean squared differences between the knowledgeables' mean estimates of service need and the target populations' expressed need were used to examine the validity of the survey of knowledgeables needs assessment technique when applied to a rural population.

Three of the 15 Spearman rank order correlation coefficients were significant at the .10 level or better: Grainger, Hamblen and Union Counties. Upon close examination no distinct similarities were found setting these three counties apart from the remaining 12.

When mean squared differences of service need were calculated and grouped by knowledgeable occupational category, knowledgeables working in primary health care clinics, planning agencies and social service agencies were found to be the best predictors of need. A limitation associated with this finding was cited as the sample size of primary health care clinics was very small and the appropriateness of generalizing to a larger population was questioned.

Overall, the comparative analysis showed little relationship between the expressed service needs of the target population and the mean estimates determined by the survey of knowledgeables. This discrepancy brought about questions of the validity of the survey of knowledgeables when applied to a rural population.