Multimodal neurosurgery force feedback system based on mesh fusion modeling

Virtual reality based force feedback system is spotlighted as a safe and efficient training environment to obtain surgical skills. Neurosurgery utilizes multimodal patient images for visualization of a variety of functions in head. The aim of this study is to establish a concept of multimodal neurosurgery force feedback system based on mesh fusion modeling. In the model of mesh fusion, we developed an algorithm to detect overlapped region between the multiple meshes that are obtained from multimodal images, and to determine a new boundary between the meshes. Then, the method solved interaction between the newly defined mesh boundaries using the interaction model based on a finite element method. The proposed method was implemented, and applied to both simple and patient datasets for evaluating its applicability. As a result, the method succeeded to be applied to both simple and patient datasets. Finally, we demonstrated the early stage of the surgical approach in neurosurgery. Simulation results showed a real-time simulation of brain tissue deformation with force feedback.