Improving Speed of Generalized Linear Models (GLMs) in R

Generalized linear models (GLMs) are widely used in statistical modeling to analyze data that do not follow a normal distribution. However, fitting multiple GLMs can be computationally expensive, particularly when dealing with large datasets. In this article, we will explore ways to improve the speed of GLM fitting using the fastglm and speedglm packages in R.

Introduction

The IRLS (Iteratively Reweighted Least Squares) algorithm is typically used for fitting GLMs, which requires matrix inversion/decomposition at each iteration. This can be computationally expensive, especially when dealing with large datasets. The fastglm package offers several options to improve the speed of GLM fitting.

Overview of GLM Fitting Algorithms

GLM fitting algorithms typically use the IRLS algorithm, which requires matrix inversion/decomposition at each iteration. The IRLS algorithm works by iteratively reweighting the data points and updating the model parameters until convergence. This process can be computationally expensive, especially when dealing with large datasets.

Using fastglm for Faster GLM Fitting

The fastglm package offers several options to improve the speed of GLM fitting. The default choice is a slower but more stable option (QR with column-pivoting), which can be replaced by one of two available Cholesky-type decompositions to improve speed dramatically.

Cholesky-Type Decompositions in fastglm

The method argument in fastglm allows one to change the decomposition. Option 2 gives the vanilla Cholesky decomposition, while option 3 gives a slightly more stable version of this.

library(fastglm)
# fit GLM using fastglm with default method (QR with column-pivoting)
system.time(m_glm <- glm(fo, data=df, family = binomial))

# fit GLM using fastglm with Cholesky-type decomposition option 2
system.time(m_fastglm_2 <- fastglm(x, y, family = binomial(), method = 2))

# fit GLM using fastglm with Cholesky-type decomposition option 3
system.time(m_fastglm_3 <- fastglm(x, y, family = binomial(), method = 3))

Comparison of Timings

The timings for the provided example are:

• m_glm: 23.206 seconds
• m_speedglm: 15.448 seconds
• m_fastglm_2: 2.159 seconds
• m_fastglm_3: 2.247 seconds

As shown, using Cholesky-type decompositions in fastglm can significantly improve the speed of GLM fitting.

Using speedglm for Faster GLM Fitting

The speedglm package also offers several options to improve the speed of GLM fitting. One notable difference between speedglm and standard IRLS implementations is its careful use of half-steps to prevent divergence.

Half-Steps in speedglm

speedglm uses half-steps to prevent divergence, which can be more numerically stable than using full steps as used in standard IRLS implementations. However, this comes at the cost of increased computational expense.

library(speedglm)
# fit GLM using speedglm with default settings
system.time(m_speedglm <- speedglm(fo, data=df))

# fit GLM using speedglm with half-steps enabled
system.time(m_speedglm_half <- speedglm(fo, data=df, half.step = TRUE))

Conclusion

In conclusion, both fastglm and speedglm offer several options to improve the speed of GLM fitting in R. By choosing the most suitable decomposition option and using half-steps when necessary, one can significantly reduce the computational expense associated with GLM fitting.

Code Snippet

Here is a code snippet that demonstrates how to use fastglm and speedglm for faster GLM fitting:

library(fastglm)
library(speedglm)

# create data
set.seed(123)
x <- rnorm(1000)
y <- rnorm(1000, mean = 2, sd = 1.5)
fo <- glm(y ~ x, family = binomial)

# fit GLM using fastglm with default method (QR with column-pivoting)
system.time(m_glm <- glm(fo, data=df))

# fit GLM using fastglm with Cholesky-type decomposition option 2
system.time(m_fastglm_2 <- fastglm(x, y, family = binomial(), method = 2))

# fit GLM using speedglm with default settings
system.time(m_speedglm <- speedglm(fo, data=df))

# fit GLM using speedglm with half-steps enabled
system.time(m_speedglm_half <- speedglm(fo, data=df, half.step = TRUE))

Note: This code snippet assumes that the fastglm and speedglm packages are installed and loaded in R.

Last modified on 2025-04-20