Virtual Reality for Surgery: Key Aspects
ScienceSoft has 15 years of experience in software development for the healthcare industry, and 24 years in 3D modeling combined with experience in VR development.
Market Info: Virtual Reality in Healthcare
The worldwide market of virtual reality in healthcare has reached $336.9 million in 2020. It is forecasted to grow at a CAGR of 30.7% from 2020 till 2027. The surgery field is one of the fastest VR adopters due to the need for more efficient training of students and residents.
How Virtual Reality for Surgery Works
VR system architecture
The client VR application extracts the data from the database web server and outputs it on a user’s visualizer (a head-mounted display (HMD) or a desktop). The web admin panel is used for database management and settings access.
Surgery simulation for immersive training and surgeons’ skills assessment. For example, if a trainee needs to remove a tumor, the script of the simulation describes where it is located, how it looks, medical device readings and how they change after each step of the operation protocol is completed.
Patient data (e.g., MRI, CT) is converted into 3D models. A surgeon can navigate the models (scale, rotate, mark, etc.) in VR to plan a surgical intervention and prepare a patient.
Medical device companies can use VR software to let the surgeons try new products in the virtual environment.
Features inherent to all use cases:
Users can collaborate as if they were in one operating room, for example, hand over surgical instruments.
The surgery simulation 3D recording can be replayed to show the patients the plan of an upcoming surgery or to analyze a trainee’s performance.
The feature can be used by teachers for guided training, by surgeons for remote case discussion, or by medical sales reps to guide surgeons on the usage of new medical devices in VR demo.
3D models of virtual patients (each with the unique patient data, response of the patient’s organs), surgical instruments, an operating room layout, etc.
Features specific to surgical training:
The difficulty of certain tasks in the surgery simulation can be adjusted based on the skill level of a trainee.
VR system analyses the actions of a trainee in comparison to performance metrics, gives feedback on their skills, and tracks their progress over time.
Important Challenges and Solutions
Virtual reality for surgery simulation should render the interaction of surgical instruments with tissue structures with high precision. A user should receive a real-life response to their manipulations.
With a multi-point collision detection method, a group of sample points is placed evenly across the surface of the surgical instruments. The 3D model of an operated organ is built with voxels. When sample points of the instruments come in contact with voxels, the system computes the applied force in each sample point and deletes the affected voxels, thus accurately deforming the tissue.
Virtual reality creates the environment to practice skills but can’t assess them in detail.
Integration of AI in a VR system to measure trainees’ performance. The data for performance benchmarks (e.g., a surgical instrument’s position, the force applied to an instrument, timeliness, amount of the tissue removed) is derived from the users with different experience (surgeons, residents, med students) going through the same surgical procedure in a VR simulator. Then a machine learning algorithm is fed with this data to give feedback on performance to future trainees.
Note: for tasks, where human verification is still needed (e.g., medical exams), multi-player mode can be used.
General cost drivers
- Method of creating 3D models (3D CAD rendering, photo- or video shoot)
- Number of 3D models (in case of 3D CAD rendering).
- Number of user roles.
- Number and complexity of surgery simulation scripts.
Additional costs of the entire solution
- Development of an AI-based performance assessment module.
- Development of software for MRI and CT scans conversion to 3D models.
- VR hardware: PC (with high-end GPU and CPU), HMDs and controllers.
- Infrastructure services (e.g., cloud services).
VR Consulting and Development by ScienceSoft
ScienceSoft combines experience in VR and healthcare to help create innovative apps tailored to the surgery specifics.
Our consultants help:
- Decide on the system’s functionality.
- Choose the right tech stack.
- Define product development and management roadmap.
- Estimate TCO of the VR system.
- Estimate ROI.
The service covers all the stages of VR system creation:
- Business analysis.
- Architecture design.
- 3D modeling and software development.
- Testing and QA.
ScienceSoft is a technology consulting and implementation provider with 31 years of practice in IT. We help product companies across the globe implement the cutting-edge technology of virtual reality.