Publications Details
Low-Variance Hessian Estimation: Coordinating High-Dimensional Sparsification with Projective Integral Updates
We can improve network pruning by leveraging the loss-topography extraction techniques used by projective integral updates for variational inference. Low-variance Hessians facilitate more aggressive pruning by providing better loss approximations when a parameter is removed.