Researchers found a way to extract accurate 3D surgical tool movement from ordinary 2D camera feeds without expensive sensors or hardware. How?
Researchers from Indian Institute of Technology Bombay and Indian Institute of Technology Goa have developed a computational method that can track surgical instruments in three dimensions using only conventional two dimensional laparoscopic video feeds. The approach could help make advanced surgical visualisation systems more accessible to smaller healthcare facilities that cannot afford expensive robotic surgery platforms or specialised imaging hardware.
Laparoscopic surgery, often referred to as keyhole surgery, relies on surgeons operating instruments inside the body while viewing a 2D camera feed from the surgical site. Since the camera lacks natural depth perception, surgeons depend heavily on experience and spatial judgement to interpret movements in 3D space. Existing systems capable of generating 3D visualisation typically require dual camera setups, additional sensors, labelled instruments, or computationally intensive deep learning techniques, all of which significantly increase costs and complexity.
The research team instead adopted a geometry based software approach that estimates the position, orientation, and movement of surgical instruments directly from a standard monocular video feed. By modelling instruments as connected geometric structures and analysing changes in shape, angles, and size across video frames, the algorithm reconstructs depth and rotational movement in real time.
One of the key advantages of the system is its low computational requirement. The algorithm operates at nearly 50 frames per second on a standard processor without requiring specialised graphics hardware. According to the researchers, the system achieved displacement errors of less than or equal to one millimetre while maintaining highly accurate orientation tracking during simulations and physical experiments.
The method also uses interval based uncertainty modelling to improve stability in situations involving poor lighting, motion blur, or unclear instrument outlines. Instead of assigning a single exact location to the surgical tool, the algorithm calculates a probable range where the instrument tip could exist, helping reduce ambiguity in 3D estimation.
The researchers believe the technology could eventually support cost effective surgical assistance systems, virtual reality training platforms, and real time depth enhanced visualisation during minimally invasive procedures using existing laparoscopic camera setups.
Prof Leena Vachhani from the Indian Institute of Technology (IIT) Bombay, concludes, “This work demonstrates that a three-dimensional visual experience for surgeons can be achieved using the existing monocular laparoscopic camera itself, offering a cost-effective and practical pathway toward improved depth perception in minimally invasive surgery.”