Can’t utilize for Shorter Generations

[gurushop]

Wouldn’t it be possible to set when riflex jumps in? Rather than having to do a 132+ frame output, shouldn’t it be possible to set it double day a 60 frame output?

[zhuhz22]

Hi @gurushop , thank you for your attention to our work.

I understand that your issue is that "If I want to generate videos shorter than N_k, how to ban the introduction of RIFLEx in the code". Actually, you can directly set args.num_frames to your ideal num_frames, and if (args.num_frames-1)//4+1 < N_k (i.e., so short that it's unnecessary to introduce RIFLEx), then our code is equal to the baseline without RIFLEx (i.e., the original code from the official repository).

The above judgement is defined at https://github.com/thu-ml/RIFLEx/blob/c7cf51f868473ddc4c05ecac1af337970f357d4f/hunyuanvideo.py#L80

    # For training-free, if extrapolate length exceeds the period of intrinsic frequency, modify RoPE
    if L_test > args.N_k and not args.finetune:
        pipe.transformer.rope = HunyuanVideoRotaryPosEmbedRifleX(args.k, L_test, original_rope.patch_size, original_rope.patch_size_t, original_rope.rope_dim,original_rope.theta)

So only if L_test > args.N_k will RIFLEx be applied.

[gurushop]

I suppose these issue requests are for the technically advanced. I see what you mean, but say for example on a lower VRAM card, you’re only capable of generating a 61F output without overloading the memory getting an OOM.

Am I understanding it correctly that the use of riflex could be set to double that output at the same VRAM to 122F for example?

Any way to change the setting you mentioned through comfyui or is modifying that file the only way.

Thanks.

[zhuhz22]

RIFLEx is an approach that addresses length extrapolation from the perspective of the fundamental capabilities of generative models, rather than being a work in the field of efficient machine learning. RIFLEx focuses on solving the content repetition problem faced by the baseline while consuming the same computational resources as the baseline, rather than on extending the generation length under the same resource constraints.

We might be able to tackle the efficiency issue in our future work, and thank your for proposing such a need.