Optimal monitoring systems using statistical experimental design

A statistical experimental design methodology is demonstrated for determining the optimal number and placement of data collection devices of a monitoring sys- tem for a computer network such as the Internet or its subsystems. The approach is based on Mercer's expansion of covariance kernels in the case of continuous controlled variables and on the direct use of convex design theory for discrete de- signs sets with spatial covariance structure. The merit of the methodology using Mercer's kernel expansion is illustrated with some examples based on covariance structures generated by the Brownian bridge model. The results of this dissertation establish the optimality conditions for the D- and linear criteria in the presence of a spatial covariance structure and discrete variable space. Associated algorithms and computer software are developed to implement the theory. In ad- dition to constructing optimal designs, the computer network examples exhibit the ability of the algorithms to suggest and estimate the efficiency of monitoring system designs which are less than optimal but which may be attractive according to some operational, policy, or cost perspectives.

The applicability of the methods to computer network monitoring is demonstrated with simple network graph examples for single- and multiple-host monitoring systems and with a test case from the Department of Energy (DOE) Energy Sciences Network (ESnet).

The simple network graph examples utilize the D-optimality and minimax criteria for linear models both with and without a prior information structure. The analysis is repeated for the case when the variance is dependent upon the route selected. The examples show that the transition from single-host to multiple-host monitoring systems produces a significant gain with respect to efficiency and cost reduction.

The ESnet test case is created using an Oak Ridge National Laboratory host computer that interrogates 39 sites of the ESnet host community. The developed methodology for constructing experimental designs for D- and linear optimality in the presence of a spatial covariance structure-produces an optimal monitoring system with 12 sites, which is compared with a few other designs.