Fast Kernel Learning via Kernel Conjugate Gradient Export

@article{citeulike:3740267, abstract = {We propose a novel variant of the conjugate gradient algorithm, Kernel Conjugate Gradient (KCG), designed to speed learning for kernel machines with differentiable loss functions. This approach leads to a better conditioned optimization problem during learning. We establish an upper bound on the number of iterations for KCG that indicates it should require less than the square root of the number of iterations that standard conjugate gradient requires. In practice, for various differentiable kernel learning problems, we find KCG consistently, and significantly, outperforms existing techniques. The algorithm is simple to implement, requires no more computation per iteration than standard approaches, and is well motivated by Reproducing Kernel Hilbert Space (RKHS) theory. We further show that data-structure techniques recently used to speed kernel machine approaches are well matched to the algorithm by reducing the dominant costs of training: function evaluation and RKHS inner product computation.}, author = {Ratliff and Bagnell}, citeulike-article-id = {3740267}, citeulike-linkout-0 = {http://www.cs.cmu.edu/~ndr/important-documents/papers/kcg-paper-icmlVersion.pdf}, posted-at = {2008-12-03 13:30:30}, priority = {4}, title = {Fast Kernel Learning via Kernel Conjugate Gradient}, url = {http://www.cs.cmu.edu/~ndr/important-documents/papers/kcg-paper-icmlVersion.pdf} }

TY - JOUR ID - citeulike:3740267 L3 - citeulike-article-id:3740267 N2 - We propose a novel variant of the conjugate gradient algorithm, Kernel Conjugate Gradient (KCG), designed to speed learning for kernel machines with differentiable loss functions. This approach leads to a better conditioned optimization problem during learning. We establish an upper bound on the number of iterations for KCG that indicates it should require less than the square root of the number of iterations that standard conjugate gradient requires. In practice, for various differentiable kernel learning problems, we find KCG consistently, and significantly, outperforms existing techniques. The algorithm is simple to implement, requires no more computation per iteration than standard approaches, and is well motivated by Reproducing Kernel Hilbert Space (RKHS) theory. We further show that data-structure techniques recently used to speed kernel machine approaches are well matched to the algorithm by reducing the dominant costs of training: function evaluation and RKHS inner product computation. TI - Fast Kernel Learning via Kernel Conjugate Gradient AU - Ratliff AU - Bagnell UR - http://www.cs.cmu.edu/~ndr/important-documents/papers/kcg-paper-icmlVersion.pdf ER -