Issue when seeking frames.

[sotelo]

Describe the bug

I'm getting a RuntimeError: Failed to perform color conversion. when I use the seek functionality of the decoder. I have a file that I encoded also with VPF using gop=100 as a parameter of the encoder. Then, there's this weird behaviour where if I first seek for a frame between 1 and 100 and then seek for a frame > 100, the decoding fails. Interestingly, this just happens within the first 100 frames, so if I seek for frame 105 and then for frame 205, decoding works well.

... initialize decoder
seek(frame=5) # works
seek(frame=101) # fails 

... initialize decoder
seek(frame=5) # works
seek(frame=201) # fails 

... initialize decoder
seek(frame=5) # works

... initialize decoder
seek(frame=101) # works

... initialize decoder
seek(frame=101) # works
seek(frame=5) # works 

... initialize decoder
seek(frame=105) # works
seek(frame=201) # works