-
All you need for attention
A simple learning note on attention mechanism.
-
Why is your energy spectrum incorrect?
The energy spectrum of turbulence is a crucial quantity for analyzing turbulence data. However, there are many subtleties in calculating the energy spectrum, which can lead to incorrect results if not handled properly. In this blog, we will discuss how to calculate the energy spectrum of turbulence correctly using Fourier transforms and binning techniques.
-
Why is your LoRA not memory efficient?
Although LoRA is a popular technique for finetuning, there are some subtle reasons in your LoRA implementation that might lead to increased computational overhead and memory usage. In this blog, we will explore these reasons and provide insights on how to optimize your LoRA implementation for better memory efficiency.
-
Maximum likelihood estimation, Loss functions, and regularization
From maximum likelihood estimation to loss functions and regularization in deep learning
-
Activations, Gradient Vanishing, and Weight Initialization
How activation functions affect the gradient vanishing problem and how to initialize weights properly to avoid it.