基于Transformer的图像稠密光流估计RAFT介绍

任务

输入两张RGB图像，稠密光流估计算法将输出两张图片对应像素的位移量

模型描述

本模型基于RAFT: Recurrent All-Pairs Field Transforms for Optical Flow算法。 技术细节请见：

RAFT: Recurrent All-Pairs Field Transforms for Optical Flow
Zachary Teed, Jia Deng
ECCV 2020

如何使用

代码示例(详见tests/pipelines/testdenseopticalflowestimation.py)

from modelscope.outputs import OutputKeys
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks
from modelscope.utils.test_utils import test_level

task = 'dense-optical-flow-estimation'
model_id = 'Damo_XR_Lab/cv_raft_dense-optical-flow_things'

input_location = [[path to image 1, path to image 2]]

estimator = pipeline(Tasks.dense_optical_flow_estimation, model=model_id)
result = estimator(input_location)
flow = result[0][OutputKeys.FLOWS]
flow_vis = result[0][OutputKeys.FLOWS_COLOR]

Bibtex

``` @misc{teed2020raft, title={RAFT: Recurrent All-Pairs Field Transforms for Optical Flow}, author={Zachary Teed and Jia Deng}, year={2020}, eprint={2003.12039}, archivePrefix={arXiv}, primaryClass={cs.CV} }