Skip to content

README.md

Latest commit

 

History

History
33 lines (26 loc) · 2.71 KB

README.md

File metadata and controls

33 lines (26 loc) · 2.71 KB

[TIP2021] OIFlow: Occlusion-Inpainting Optical Flow Estimation by Unsupervised Learning

Shuaicheng Liu1,2, Kunming Luo2, Nianjin Ye2, Chuan Wang2, Jue Wang2, Bing Zeng1

1. University of Electronic Science and Technology of China

2. Megvii Research

This is the official implementation of paper OIFlow: Occlusion-Inpainting Optical Flow Estimation by Unsupervised Learning, IEEE Transactions on Image Processing, 2021

inpainting_visualization

Abstract

Occlusion is an inevitable and critical problem in unsupervised optical flow learning. Existing methods either treat occlusions equally as non-occluded regions or simply remove them to avoid incorrectness. However, the occlusion regions can provide effective information for optical flow learning. In this paper, we present OIFlow, an occlusion-inpainting framework to make full use of occlusion regions. Specifically, a new appearance-flow network is proposed to inpaint occluded flows based on the image content. Moreover, a boundary dilated warp is proposed to deal with occlusions caused by displacement beyond the image border. We conduct experiments on multiple leading flow benchmark datasets such as Flying Chairs, KITTI and MPISintel, which demonstrate that the performance is significantly improved by our proposed occlusion handling framework.

Overview

overview (a) reference image. (b) ground truth flow. (c)(d) results without / with the optical flow refinement. (e) detected occlusion map. (f) a zoom-in window with mask (dark region) overlaid on the image. Here an appearance flow is learned on the image domain, which is further used to inpaint the occluded regions by the non-occluded ones. (g) the appearance flow of (f) is applied to guide the propagation in the flow field. (h) The propagated results in the zoom-in region.

Citation

If you think this work is helpful, please cite

    @article{liu2021oiflow,
      title={OIFlow: occlusion-inpainting optical flow estimation by unsupervised learning},
      author={Liu, Shuaicheng and Luo, Kunming and Ye, Nianjin and Wang, Chuan and Wang, Jue and Zeng, Bing},
      journal={IEEE Transactions on Image Processing},
      volume={30},
      pages={6420--6433},
      year={2021},
      publisher={IEEE}
    }

Acknowledgement

Part of our codes are adapted from IRR-PWC and UnFlow, we thank the authors for their contributions.