calc_optical_flow_pyr_lk not populating next_pts

[sjbeskur]

I am trying to use the lucas kanade rust bindings, but I think that I am misunderstanding the api or there is perhaps a bug:

given two very similar images the following method fails to populate next_pts
calc_optical_flow_pyr_lk_def

let mut next_pts = Vector::<Point2f>::new();
let mut err = Vector::<f64>::new();
let mut status = Vector::<u8>::new();
let _ = calc_optical_flow_pyr_lk_def(
	&img_prev ,
	&img_curr ,
	&kf_key_points ,
	&mut next_pts ,
	&mut status ,
	&mut err ,
);

status vec returns all 1's

next_pts is indeed resized to the expected new size however the values are populated only with 0.0's

the corrollary python bindings work as expected.

Any guidance would be greatly appreciated.

[sjbeskur]

I have added a "unit test" and a corresponding python notebook to highlight the differences in the results and have submitted a PR here You don't really have to merge this but I'm not seeing the solution.

The code can be found in my forked branch here: (branch unit/opt_flow)
https://github.com/sjbeskur/opencv-rust/blob/unit/opt_flow/tests/optflowpyrlk.rs

Test images are from opencv_extra repo

I'm using the latest opencv-rust (0.92.2) built against OpenCV v4.9.0. Please let me know if there is any other information I can provide. I assume I'm just doing something incorrectly but out of ideas.

[twistedfall]

I'll check it out, thank you for all the preparation work!

[twistedfall]

The example in your PR is actually failing, when I replace let _ = with .unwrap()s I'm getting:

called `Result::unwrap()` on an `Err` value: Error { code: "StsAssert, -215", message: "OpenCV(4.10.0) /home/pro/projects/opencv-lib/opencv-4/opencv-4.10.0/modules/core/src/matrix_wrap.cpp:1393: error: (-215:Assertion failed) mtype == type0 || (CV_MAT_CN(mtype) == CV_MAT_CN(type0) && ((1 << type0) & fixedDepthMask) != 0) in function 'create'\n" }

on the line that calls calc_optical_flow_pyr_lk. That most probably indicates that there is something wrong with the input arguments. I'll try to investigate further what exactly is wrong.

[sjbeskur]

yep I get the same only using OpenCV(4.9.0). I should have caught this earlier but still not clear what input i wrong.

[twistedfall]

I think I have it, can you try changing the type of let mut err to Vector<f32>?

	let mut err = Vector::<f32>::new();

[sjbeskur]

Yep that looks like it did the trick. Thank you so much for the help and terribly sorry for not seeing it myself. I'll update my "example" / unit test in case anyone else encounters the issue.