O(N2)-Operation approximation of covariance matrix inverse in Gaussian process regression based on quasi-Newton BFGS method
Leithead, W.E. and Zhang, Yunong, Science Foundation Ireland grant, 00/PI.1/C067 (Funder), EPSRC, GR/M76379/01 (Funder) (2007) O(N2)-Operation approximation of covariance matrix inverse in Gaussian process regression based on quasi-Newton BFGS method. Communications in Statistics - Simulation and Computation, 36 (2). pp. 367-380.
Full text not available in this repository.Request a copy from the Strathclyde authorAbstract
Gaussian process (GP) is a Bayesian nonparametric regression model, showing good performance in various applications. However, during its model-tuning procedure, the GP implementation suffers from numerous covariance-matrix inversions of expensive O(N3) operations, where N is the matrix dimension. In this article, we propose using the quasi-Newton BFGS O(N2)-operation formula to approximate/replace recursively the inverse of covariance matrix at every iteration. The implementation accuracy is guaranteed carefully by a matrix-trace criterion and by the restarts technique to generate good initial guesses. A number of numerical tests are then performed based on the sinusoidal regression example and the Wiener-Hammerstein identification example. It is shown that by using the proposed implementation, more than 80% O(N3) operations could be eliminated, and a typical speedup of 5-9 could be achieved as compared to the standard maximum-likelihood-estimation (MLE) implementation commonly used in Gaussian process regression.
Creators(s): | Leithead, W.E. and Zhang, Yunong; Science Foundation Ireland grant, 00/PI.1/C067 (Funder), EPSRC, GR/M76379/01 (Funder); | Item type: | Article |
---|---|
ID code: | 11930 |
Keywords: | Gaussian process regression, matrix inverse, optimization, O(N2) operations, quasi-Newton BFGS method, Electrical engineering. Electronics Nuclear engineering, Modelling and Simulation, Statistics and Probability |
Subjects: | Technology > Electrical engineering. Electronics Nuclear engineering |
Department: | Faculty of Engineering > Electronic and Electrical Engineering |
Depositing user: | Strathprints Administrator |
Date deposited: | 14 Apr 2011 09:26 |
Last modified: | 20 Jan 2021 17:56 |
URI: | https://strathprints.strath.ac.uk/id/eprint/11930 |
Export data: |