A Computer Vision Method For Recovering 3D Structure
Of Objects Using Dual Meshes

TUBITAK Career Project 105E097 [2006 - 2009]

Project Team : Tarkan Aydin, Ulas Vural and Dr. Yusuf Sinan Akgul.

The aim of this project is to recover the 3D structure of objects via dual meshes. The dual meshes method was first proposed by Akgul and Kambhamuttu in 1999. The method was successfully applied to the biomedical video analysis. In this project, a combination of diffrent techniques such as stereo, focus and defocus is going to be used in a dual meshes framework.

The system uses two separate optimization processes for the recovery of 3D surfaces with depth discontinuities and occlusions. The optimization processes are based on gradient descent heuristics, which do not guarantee optimality. However, due to the interaction between the optimization processes, the overall result of our system is always better than the results achievable by a single optimization process. The presented idea is applicable to other heuristic optimization methods that use a global approach.

The synchronous optimization processes was formulated by introducing Etns term. This energy functional will produce two minimal cost disparity surface D1 and D2 from two separate but dependent optimizations.

These disparity surface constructed by assigning a disparity value for each element in the left image of the stereo pair.

Fig 1. Synchronous evolution of processes for baseball image (Fig.2). (a) Initial state of processes. One is started from minimum disparity values and the other is stared from maximum disparity values. (b-d) The processes are evolving to recover the same surface. (e) The final state of the processes. Both recovered the same surface.

We are currently working on elimination of the surface continuity constraint by developing a novel regularization method based on diffusion process. The handling of the discontinuities and occlusions are done by using diffusive-like regularization mechanism in which flow contributions are adjusted by the segment information and the temporal positions of the processes.

Fig 2. Initial result of synchronous optimization process with segment based regularization method, (a) Left image of baseball stereo pairs. (b) Computed depth map (c) synthesized view. Depth discontinuities are recovered well.




Fig 3. Depth maps computed by our new approach for middlebury data sets.