-
Why your LoRA is not memory efficient?
Although LoRA is a popular technique for finetuning, there are some subtle reasons in your LoRA implementation that might lead to increased computational overhead and memory usage. In this blog, we will explore these reasons and provide insights on how to optimize your LoRA implementation for better memory efficiency.
-
Maximum likelihood estimation, Loss functions, and regularization
From maximum likelihood estimation to loss functions and regularization in deep learning
-
Introduction of Fourier Spectral Method
A concise introduction to the Fourier spectral method for solving PDEs, including the basic idea of the spectral method, how to calculate linear and nonlinear terms in the spectral space, time integration methods, and the aliasing error caused by discretization.
-
Towards Conflict-free training [ICLR 2025 Spotlight]
A brief introduction on Conflict-free Inverse Gradients Method