I need to build a PC to train SDXL using Kohya (and generate a video using warpfusion). The machine we currently have uses an A5000 with 24GB of VRAM - however, it's been kinda cramped for our training (that's a whole other story) - and we share it with others. I need at least 30-32 GB of VRAM to avoid spilling into shared memory (which TANKS performance), but I also wanna keep the budget reasonable

Does it make sense to replace an A5000 with two A4000s or A4500s? Using Accelerate we should be able to combine them and get more than enough memory, but I need to know if they'd be too inferior to the A5000 and by how much

EDIT

Additional questions: does this make sense in general? are there GPUs more sutited to work in parallel/train SDXL?