Multiple Non-Rigid Surface Detection and Registration

Yi Wu¹ Yoshihisa Ijiri² Ming-Hsuan Yang¹

¹University of California, Merced ²OMRON Corporation

{ywu29,mhyang}@ucmerced.edu, ykoni@ari.ncl.omron.co.jp

Presented at ICCV 2013 [paper] [supp]

Abstract

Detecting and registering nonrigid surfaces are two important research problems for computer vision. Much work has been done with the assumption that there exists only one instance in the image. In this work, we propose an algorithm that detects and registers multiple nonrigid instances of given objects in a cluttered image. Specifically, after we use low level feature points to obtain the initial matches between templates and the input image, a novel high-order affinity graph is constructed to model the consistency of local topology. A hierarchical clustering approach is then used to locate the nonrigid surfaces. To remove the outliers in the cluster, we propose a deterministic annealing approach based on the Thin Plate Spline (TPS) model. The proposed method achieves high accuracy even when the number of outliers is nineteen times larger than the inliers. As the matches may appear sparsely in each instance, we propose a TPS based match growing approach to propagate the matches. Finally, an approach that fuses feature and appearance information is proposed to register each nonrigid surface. Extensive experiments and evaluations demonstrate that the proposed algorithm achieves promising results in detecting and registering multiple non-rigid surfaces in a cluttered scene.


(a) Templates.	(b) Detection and registration results. The blue curves indicate the top of the instances in accordance with the corresponding templates.

Registration Improvement by Match Growing

In (a) a template is shown in the bottom right of the input image. One instance is taken as an example and the matched feature points are marked as red. The non-rigid warps estimated by feature-based approach and fusion-based approach based on the feature matches shown in (a) are shown in (b) and (c), respectively. In (d) the matched feature points after match growing are marked as red points. The warps estimated by feature-based and fusion-based approaches based on the feature matches are shown in (e) and (f), respectively.

Detection Performance using Synthetic Image


(a)	(b)	(c)	(d)	(e)
(a) Precision of detection. (b) Recall of detection. (c) Precision of matches. (d) Recall of matches. (e) Synthetic image.

Outlier Rejection Results


(a)	(b)	(c)	(d)	(e)	(f)
(a) Performance of AOR with different numbers of inliers. (b) Performance of AOR with different outlier ratios. (c) Comparison of AOR and LPOR. (d) Comparison of AOR and FAOR. (e) Matches before outlier rejection. (f) Refined matches by AOR.