基于Transformer的图像局部特征匹配LoFTR介绍

任务

输入两张RGB图像，局部特征匹配算法将输出两张图片对应像素的位置

模型描述

本模型基于LoFTR: Detector-Free Local Feature Matching with Transformers算法。 技术细节请见：

LoFTR: Detector-Free Local Feature Matching with Transformers
Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, Xiaowei Zhou
CVPR 2021

如何使用

代码示例(详见tests/pipelines/testimagematching.py)

from modelscope.outputs import OutputKeys
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks


task = 'image-local-feature-matching'
model_id = 'damo/cv_resnet-transformer_local-feature-matching_outdoor-data'

input_location = [[path to image 1, path to image 2]]

estimator = pipeline(Tasks.image_local_feature_matching, model=model_id)
result = estimator(input_location)
kpts0, kpts1, conf = result[0][OutputKeys.MATCHES]
print(f'Found {len(kpts0)} matches')

Bibtex

@inproceedings{DBLP:conf/cvpr/SunSWBZ21,
  author       = {Jiaming Sun and
                  Zehong Shen and
                  Yuang Wang and
                  Hujun Bao and
                  Xiaowei Zhou},
  title        = {LoFTR: Detector-Free Local Feature Matching With Transformers},
  booktitle    = {{IEEE} Conference on Computer Vision and Pattern Recognition, {CVPR}
                  2021, virtual, June 19-25, 2021},
  pages        = {8922--8931},
  publisher    = {Computer Vision Foundation / {IEEE}},
  year         = {2021},
  url          = {https://openaccess.thecvf.com/content/CVPR2021/html/Sun\_LoFTR\_Detector-Free\_Local\_Feature\_Matching\_With\_Transformers\_CVPR\_2021\_paper.html},
  doi          = {10.1109/CVPR46437.2021.00881},
  timestamp    = {Sat, 30 Sep 2023 09:38:22 +0200},
  biburl       = {https://dblp.org/rec/conf/cvpr/SunSWBZ21.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}