Automatic Selection of Optimal Configuration of Artificial Neural Networks
Vivek Dua
University College London
Objectives: A novel mixed-integer nonlinear programming (MINLP) based mathematical formulation for artificial neural network (ANN) analysis which minimises the complexity of the configuration of the network is presented.
Methods: Traditionally, the configuration of the ANN, which comprises of the number of hidden layers, number of nodes in the hidden layers and the interconnections between the nodes, is fixed and then an optimisation problem is solved to minimise the error between the observed data and model predictions. A reduction in the number of hidden layers and nodes can result in an increase in the error whereas increasing the number of hidden layers and nodes can lead to over-fitting or over-learning [3]. A reduction in the number of interconnections can reduce the degeneracy in the input-output relationships and may also provide some insight into the behaviour of the model. The configuration is iteratively changed until an acceptable error is obtained. Automatic selection of the optimal configuration can be mathematically formulated by introducing 0-1 binary variables to represent existence or not of a layer, node or interconnection. This results in an MINLP where the objective is to minimise the complexity of the network subject to equality constraints, representing the transformations across the network, and inequality constraints, such as, an upper bound on the error [1]. The solution of this problem is given by an optimal network configuration which meets the error criteria as well as any other constraints included in the formulation.
Results: This methodology was applied to a kinetic and dynamic dataset available in open literature for a CNS compound [2]. A good comparison between the observed and predicted data was obtained.
Conclusions: Redundant nodes, layers and interconnections are eliminated and a compact representation of the input-output correlations is obtained.
References:
[1] Dua, V. (2006) Optimal configuration of artificial neural networks, Proc. of 16th ESCAPE and 9th Int Symp on PSE, W. Marquardt and C. Pantelides (Eds), p.1599-1604, Elsevier, Amsterdam.
[2] Gabrielsson, J. and Weiner, D. (2000) Pharmacokinetic and pharmacodynamic data analysis: concepts & applications, 3rd ed., Taylor and Francis, p829.
[3] Gobburu, J.V.S. and Chen, E.P. (1996) J. Pharm Sci, 85, 505-510.
