en flag +1 214 306 68 37

Virtual Reality for Surgery: Key Aspects

VR for surgery - ScienceSoft

ScienceSoft has 19 years of experience in software development for the healthcare industry, and 28 years in 3D modeling combined with experience in VR development.

Overview of Virtual Reality for Surgery

Used for surgery planning, medical training, etc., VR solutions help surgeons by generating three-dimensional images of anatomical structures and providing them with tools to simulate a surgery in a safe environment. Research finds that VR solutions tend to improve surgery outcomes and patient safety.

Market Info: Virtual Reality in Surgery

The worldwide market of virtual reality in healthcare has reached $628 million in 2022. It is forecasted to grow to $6.2 billion in 2029 at a CAGR of 38.7%. The surgery field is one of the fastest VR adopters due to the need for more efficient training of students and residents.

How Virtual Reality for Surgery Works

VR system architecture

The client VR application extracts the data from the database web server and outputs it on a user’s visualizer (a head-mounted display (HMD) or a desktop). The web admin panel is used for database management and settings access.

VR system architecture

Use cases

Surgical training

Surgery simulation for immersive training and surgeons’ skills assessment. For example, if a trainee needs to remove a tumor, the script of the simulation describes where it is located, how it looks, medical device readings and how they change after each step of the operation protocol is completed.

Pre-surgery planning

Patient data (e.g., MRI, CT) is converted into 3D models. A surgeon can navigate the models (scale, rotate, mark, etc.) in VR to plan a surgical intervention and prepare a patient.

Medical marketing

Medical device companies can use VR software to let the surgeons try new products in the virtual environment.


Features inherent to all use cases:

Multi-user mode

Users can collaborate as if they were in one operating room, for example, hand over surgical instruments.

Simulation recording

The surgery simulation 3D recording can be replayed to show the patients the plan of an upcoming surgery or to analyze a trainee’s performance.

Remote telepresence

The feature can be used by teachers for guided training, by surgeons for remote case discussion, or by medical sales reps to guide surgeons on the usage of new medical devices in VR demo.

Assets libraries

3D models of virtual patients (each with the unique patient data, response of the patient’s organs), surgical instruments, an operating room layout, etc.

Features specific to surgical training:

Scripts editor

The difficulty of certain tasks in the surgery simulation can be adjusted based on the skill level of a trainee.

Performance tracking and assessment

VR system analyses the actions of a trainee in comparison to performance metrics, gives feedback on their skills, and tracks their progress over time.

Success Stories of VR in Surgery

Johnson and Johnson Medical Devices Companies developed a VR training system for practicing Anterior approach in hip surgery. The Imperial College London’s later study showed that 83% of the surgeons, who completed this VR training, could successfully perform the real surgery with minimal guidance, whereas those, who didn’t complete the training, required assistance.

ImmersiveTouch, the startup offering an FDA-cleared surgical planning platform ImmersiveView, reached $5 million revenue in 2021. The latest version of the tool continues to set the standard in surgical planning by generating high-fidelity 3D replicas from patient DICOM scans and enabling doctors to perform a variety of actions using Oculus Quest 2 hand controllers.

Want to Create a Virtual Reality Application for Surgery?

ScienceSoft’s team is ready to take up your idea all the way from shaping a user-centered product vision to the development of a reliable VR application for surgery.


Important Challenges and Solutions

Challenge #1

Virtual reality for surgery simulation should render the interaction of surgical instruments with tissue structures with high precision. A user should receive a real-life response to their manipulations.

Check out the solution


With a multi-point collision detection method, a group of sample points is placed evenly across the surface of the surgical instruments. The 3D model of an operated organ is built with voxels. When sample points of the instruments come in contact with voxels, the system computes the applied force in each sample point and deletes the affected voxels, thus accurately deforming the tissue.


Challenge #2

Virtual reality creates the environment to practice skills but can’t assess them in detail.

Check out the solution


Integration of AI in a VR system to measure trainees’ performance. The data for performance benchmarks (e.g., a surgical instrument’s position, the force applied to an instrument, timeliness, amount of the tissue removed) is derived from the users with different experience (surgeons, residents, med students) going through the same surgical procedure in a VR simulator. Then a machine learning algorithm is fed with this data to give feedback on performance to future trainees.

Note: for tasks, where human verification is still needed (e.g., medical exams), multi-player mode can be used.


Costs of VR for Surgery

General cost drivers

  • Method of creating 3D models (3D CAD rendering, photo- or video shoot)
  • Number of 3D models (in case of 3D CAD rendering).
  • Number of user roles.
  • Number and complexity of surgery simulation scripts.

Additional costs of the entire solution

  • Development of an AI-based performance assessment module.
  • Development of software for MRI and CT scans conversion to 3D models.
  • VR hardware: PC (with high-end GPU and CPU), HMDs and controllers.

Operational costs

  • Infrastructure services (e.g., cloud services).

From ScienceSoft’s experience in the domain, a virtual reality application for surgery may cost from $200,000 to $300,000.

VR Consulting and Development by ScienceSoft

ScienceSoft combines experience in VR and healthcare to help create innovative apps tailored to the surgery specifics.

VR consulting

Our consultants help:

  • Decide on the system’s functionality.
  • Choose the right tech stack.
  • Define product development and management roadmap.
  • Estimate TCO of the VR system.
  • Estimate ROI.
Request VR consulting

VR development

The service covers all the stages of VR system creation:

  • Business analysis.
  • Architecture design.
  • 3D modeling and software development.
  • Testing and QA.
Request VR development

About ScienceSoft

ScienceSoft is a technology consulting and implementation provider with 35 years of practice in IT. Being ISO 13485-certified, we design and develop medical software according to the requirements of the FDA and the Council of the European Union and help product companies across the globe implement the cutting-edge technology of virtual reality. Our top priority is driving project success no matter what while keeping to the agreed time and budget and responding to uncertainties agilely.