About GPU requirements for training

[Tommy-YQ]

Hi authors,

Thanks for your impressive work. The paper states training was performed on 32 GPUs with a global batch size of 32, yet details regarding per-GPU VRAM requirements are not specified. Could you kindly share the minimum GPU memory specification needed to reproduce your training setup?

Thanks in advance.

[kj-chen666]

Hi,

Thanks for your interest in our work. We used 32 GPUs with 80GB VRAM each for training.

Best regards,
KJ