Evaluation methods for Internet management schemes

This research seeks to identify a systematic procedure to evaluate network management frameworks. The methods developed here are based on the need to make a comparative evaluation of the ISO and Internet network management frameworks, as specified by the Common Management Information Protocol (CMIP) and the Simple Network Management Protocol (SNMP) respectively. The basic principle underlying the synthesis of these evaluation procedures is the separation of the services provided by the management framework into groups that are likely to exhibit different characteristics with reference to the evaluation measures. It is expected that these procedures can also be applied to other management frameworks that employ a query response model. However, the relative evaluation of two management protocols is only possible to the extent they support common sets of management operations.

The evaluation of the physical, data link, network, and transport protocols has received considerable attention in the literature. In contrast, the performance analysis of the upper layers (session, presentation, and application) has remained virtually untouched. Since network management protocols reside in the application layer, this work represents one of the first attempts to evaluate an upper layer protocol.

CMIP has been implemented over the ISO Development Environment (ISODE) platform as a part of this project. The evaluation process covers the agent's performance and host impact, as well as the management framework's response in lossy environments, and its impact on the network. Stochastic models have been introduced to track the behavior of the connection oriented and connectionless network management schemes in lossy environments.

Application of the evaluation methods to CMIP and SNMP have led to a number of interesting results. It is shown that scoping in CMIP can be CPU intensive unless a proper scoping method is chosen. Using the Remote Operations Service of ISO requires a large amount of code, especially since the RO specification of CMIP is complex. The "semantic gap" between the physical encoding of data and that which can be conveniently used by the management agent at the application layer leads to a large amount of code for translating back and forth between the two representations. It is shown that the connectionless approach is necessary for network management when packets are dropped. It is also found that the linked reply facility of CMIP minimizes network impact when the network is not lossy. Recommendations are provided for the construction of future network management protocols.