Fully fused SwiGLU? #39
Description
Description
Is there a sample for a fully fused SwiGLU layer? I see that in tile gym there is a fused_swiglu.py example, but that is partially fused using a cuTile matmul and silu_and_mul.
{{ message }}
Is there a sample for a fully fused SwiGLU layer? I see that in tile gym there is a fused_swiglu.py example, but that is partially fused using a cuTile matmul and silu_and_mul.