Feature Request: ROCm Backend

[IIIIIllllIIIIIlllll]

May I ask whether there are any plans to add a ROCm backend? This would be extremely useful for AMD GPUs!

I don't have much knowledge of C/C++ development, nor do I understand CUDA or ROCm. As a tentative experiment, I asked DeepSeek V4 PRO to generate a ROCm version, and it did compile and run successfully on my Linux + Strix Halo. However, it didn't work on Windows with a 780M.

I'm fully aware that, given my lack of relevant expertise, the code produced by AI is most likely subpar – no offense intended. This is simply the plea of an AMD user who truly hopes to see native support.

[0xShug0]

@IIIIIllllIIIIIlllll Thank you for testing! The challenge for me is that I don’t have access to ROCm machines, so I can’t really test it myself. One big lesson I’ve learned is that even if something compiles on a given backend, it doesn’t necessarily mean it can run, due to ggml backend support or other caveats. It’s not uncommon for some of my test cases to fail to finish end-to-end on Vulkan or Metal while working fine on CUDA, so I often need to update the code to handle backend-specific issues. That said, would you be interested in submitting a PR to add a dedicated Makefile (AI code is fine) for the ROCm backend, plus a doc under docs/ noting which models you tested and which ones currently work?

[IIIIIllllIIIIIlllll]

@0xShug0
Created :)
Currently tested with Qwen3-TTS and Index‑TTS2. More models to follow – documentation pending broader validation.
#48

[0xShug0]

@IIIIIllllIIIIIlllll Thanks!