Objectives

To provide an interactive link between the musculo-skeletal model and the surgeon. Using virtual reality algorithms the surgeon should be able to modify the musculo-skeletal model of the patient to simulate his operative plan. The module should allow 3-D visualization and modification of the parameterized musculo-skeletal system of the patient. Musculo-skeletal model interaction will be performed by standard input devices (e.g. mouse, keyboard or simple manipulators) or advanced manipulators (e.g. six degrees of freedom with force feedback).

Progress

Planning surgical interventions in Virtual Reality

One of the tools developed in the TLEMsafe project is a system that enables the surgeons to plan a surgical procedure on the patient-specific musculoskeletal model, send the modified model to the biomechanical analysis module, and export the scenario parameters to the surgical navigation system for use in the operating room.

The functionality of the planning system is aimed at meeting the requirements of surgery types that had been selected as the most important for the TLEMsafe project:

• Tumor surgery, requiring such functionalities as osteotomies, removal of muscles, tendons and bony structures;
• Bony corrections, requiring such functionalities as multiple osteotomies and repositioning of muscle insertion sites.

The surgeon-model interface developed provides the operator with required functionalities accessible in ways that are intuitive to the user or the ways that he is used to (Figure 1).

Figure 1 - Surgery planning system: musculo-skeletal model of lower extremities (left); a close-up of the pelvis region (right).

After loading the required musculo-skeletal model, the planning software enables the user to inspect it in 3-D from any direction. The bones may be cut with a virtual cutting blade. A cut fragment of the bone may be removed and replaced by an implant or manipulated in order to better function in the musculoskeletal system. Muscle attachment points may be repositioned and whole muscles may be removed if necessary.

An exemplary simulation of a surgery aimed at increasing the acetabular coverage is shown in Figure 2. First, the surgeon inspects the 3-D model. Then, he plans three cuts (triple osteotomy). Next, he manipulates the bony fragment in order to get a better bony coverage of the acetabulum over the femoral head. The current coverage may be evaluated any time. All changes committed to the model (the surgery scenario) may be stored for biomechanical analysis in AnyBody Modeling System™ application. The parameters of osteotomies performed in the scenario may be exported to the Brainlab surgical navigation module in order to be used during the actual surgery.

Figure 2 - Visual inspection of the pelvic area with use of bone semi-transparency effect (left). The new position of acetabular fragment after planned osteotomy (right).

In order to make surgeon’s work efficient, the planning system takes advantage of novel VR instrumentation, such as 3-D displays and haptic devices. However, to ensure better accessibility, the system may be operated with use of standard input devices, like keyboard and mouse. The simplest hardware set-up of the planning system utilizes a standard computer keyboard and mouse as the input devices while the image is presented on a 2-D display. The use of a web browser with a freeware 3DVIA player limit the additional software cost to the license of the AnyBody Modeling System™ (demo and academic licenses are available). However, even on a station without AMS™ license installed the surgeon-model interface may be used with limited functionality.

Another version of the interface uses a touch screen as a single device for both input and output information from and to the user (Figure 3). It is meant to be used especially in environments where the use of other interface devices may be not recommended, such as operating rooms.

Figure 3 - Inspection of a 3-D pelvis model with a touchscreen (application launched in a Mozilla Firefox browser window).

The planning software may take advantage of modern devices in order to offer more immersive VR experience (Figure 4). Equipped with a 3-D display, it allows the operator to perceive the surgery planning environment in a natural way. An intuitive way of 3-D objects manipulation with use of a haptic device is provided as well. It has been observed that a 3-D haptic device is well-suited for operations of manipulating viewport or a bony fragment.

Figure 4 - Model manipulation with use of a haptic device (top). 3-D image presented with stereoscopic display set (down).

The system presented allows planning surgical interventions based on a patient-specific musculoskeletal model of lower extremities. The implemented functionality allows planning of tumor surgeries and bony corrections but the modular design allows broadening the range of applications in future. The developed surgery scenarios can be directly used by a) modeling software to simulate the functional outcome of the surgery and b) surgical navigation system if the simulated outcome is satisfactory. The interface takes advantage of semi-immersion when used with 3-D display (shutter glasses) and a haptic device.

References

Marcin Witkowski, Janusz Lenar, Robert Sitnik and Nico Verdonschot, "A virtual reality interface for pre-planning of surgical operations based on a customized model of the patient", Proc. SPIE 8289, 82890M (2012); http://dx.doi.org/10.1117/12.909857