ConectarCan we reduce the rank of a high-rank LoRA?
 | Can we lower the rank of high-rank LoRA to reduce mem requirements, have LoRAs in the same rank to enable torch.compile()? Turns out that we can! That too w/ good'ol random projections & SVD 📷 Original vs. Reduced Rank 🐶 Code, more results, detailed write up are in: https://huggingface.co/sayakpaul/lower-rank-flux-lora [link] [comments] |
