zoomerGP provides a fast and composable language to write, fit, and explore Gaussian Process models in R. The package allows users to compose a Gaussian process kernel using a consise formula syntax, and tune hyperparameters through type-2 ML or Variational Inference (SVGD). Both Gaussian and non-Gaussian Likelihoods are supported.
Kernel formulas:
ZoomerGP’s flagship feature is a tidy formula interface that allows users to cleanly specify complex Gaussian-process models. As an example, you can specify and fit a Gaussian Process with an RBF covariance function to a dataset with the following formula:
gaussian_process(y ~ rbf(x), data = data)This syntax also allows you to arbitrarily combine different kernels buy adding and multiplying them. As an example, one can try to fit a periodic trend that changes over time and a long-term trend to the data by using the following code:
gaussian_process(y ~ periodic(x) * rbf(x) + rbf(x), data = data)Each kernel can also act on multiple variables, as shown below:
gaussian_process(a ~ periodic(b,c) * rbf(d,e,f,g) + rbf(h), data = data)Avaliable Kernels:
At present the following kernels are implemented:
| Kernel | Function Name | Notes |
|---|---|---|
| Squared Exponential | rbf() | Results in a smooth, infinitely-differentiable posterior. Too smooth for most applications |
| Spectral Mixture | spectral0-5 | Spectal mixture kernel with 1,2,3,4, or 5 components. Can recover any stationary covariance function but is very expensive to fit. |
| Linear | linear() | Recovers a linear trend |
| Indicator | indicator() | Equal to if , zero otherwise. Useful when accounting for clustering. |
| If / mask | mask() | Equal of one if both and are equal to one, zero otherwise. Most useful when ‘masking’ out another kernel so it only acts on certain pairs of inputs |
| Periodic | periodic() | Periodic similarity function. Can accept as a keyword argument a vector of periods associated with each dimension. |
Fitting to Large Datasets with Sparse Approximations.
Gaussian Process Regression are computationally intensive, and generally take computations to fit. To allow the method to scale to large datasets, this package implements the projected process approximation as described in Rassmusen and Williams’s (2005) Gaussian Processes for Machine Learning. To turn this approximation on, use the sparse=T argument when fitting the gaussian processes please be aware that you may also need to change the default value of the number of inducing points (n_points) for numerical stability.
Optimization Methods
Currently, we the package supports 3 engines for hyperparameter optimization. bfgs is selected by default, but these can be changed using the training_method argument to gaussian_process function
-
bfgs- as implimented in the optim package in R. Will sometimes fail to coverge, leading to an error. -
rgenoud- as implimented in thergenoud package.This is likely the most robust optimizer as it avoids getting stuck in local minima, but also the most computationally intensive. -
coin- a learning-rate free algorithm that is implimented in Rust. Seems to work well across a variety of problems, but can get stuck in local minima.
Fitting Non-Gaussian Likelihoods with Variational Inference
zoomerGP also implements a variant of the Stein Variational Gaussian Processes algorithm to perform variational inference over hyperparameters and latent function values when the likelihood is non-Gaussian. Specifically, we use the Coin Sampling algorithm proposed by the same authors to provide a robust Variational Inference engine that does not require tuning by the user.
At the moment, inference for the following non-Gaussian likelihoods are implemented:
- logistic regression (via
the logistic_gaussian_processfunction).
Installation Instructions
Installing Rust
If your operating system or version of R is not installed, you must have the Rust compiler installed to compile this package from sources. After the package is compiled, Rust is no longer required, and can be safely uninstalled.
Installing Rust on Linux or Mac:
To install Rust on Linux or Mac, you can simply run the following snippet in your terminal.
Installing Rust on Windows:
To install Rust on windows, you can use the Rust installation wizard, rustup-init.exe, found at this site. Depending on your version of Windows, you may see an error that looks something like this:
In this case, you should run rustup install stable-x86_64-pc_windows-gnu to install the missing toolchain. If you’re missing another toolchain, simply type this in the place of stable-x86_64-pc_windows-gnu in the command above.
Installing Package from Github:
Once you have rust installed Rust, you should be able to install the package with either the install.packages function as above, or using the install_github function from the devtools package or with the pkg_install function from the pak package.
## Install with devtools
# install.packages("devtools")
devtools::install_github("beniaminogreen/GPR")
## Install with pak
# install.packages("pak")
pak::pkg_install("beniaminogreen/GPR")