Machine Vision Model PoC for Surgical Items Detection with 95%+ Accuracy

About Our Customer

The Customer is an innovative healthcare sustainability startup launched by one of the top US universities.

Dealing with Low Sustainability and Overspending in Perioperative Care

A scientific study conducted by the Customer found that surgical departments spend enormous financial resources on single-use sterile items, such as suture needles, laparoscopic trocars, and cautery devices. Often, these tools are opened on the instrument table but left unused during the surgery. This results in unnecessary waste of supplies since it is mandatory that all single-use items, both used and unused, are disposed of at the end of an operation.

The Customer was looking for a vendor with a deep understanding of perioperative care to develop a proof of concept (PoC) for a machine vision model. The model was to help reduce perioperative waste, thereby contributing to healthcare sustainability and cost reduction.

Localizing, Detecting, Tracking, and Classifying Surgical Items

Trusting ScienceSoft’s experience in healthcare IT since 2005 and in image analysis since 2013, the Customer turned to us for PoC development. The PoC was to test the technical feasibility of using machine vision to identify and track the use of sterile surgical instruments during operations. The ultimate goal was to provide actionable reports to perioperative administrators and surgeons to enable the design of more cost-efficient and sustainable surgical kits.

A senior business analyst and a lead data science developer joined the project on ScienceSoft’s side. Our specialists relied on the Scrum methodology to ensure project management flexibility and timely product delivery. The project was finished in three iterations, each lasting 3–4 weeks.

The delivered PoC demonstrated the following functionality:

• The localization module locates the main items on the instrument table. These are usually single-use surgical supplies like staplers, implantable devices, bins, gloves, etc. As soon as the model identifies the location of an item, it draws a bounding box around it. This allows for early object classification.
• The detection module finds all the items spread on the table by defining their boundaries. As a result, the model can distinguish between single-use sterile supplies and reusable instruments. This enables it to focus on the disposable surgical tools, as they are primarily responsible for operating room waste and surgical department expenditure. During the PoC development, ScienceSoft’s team placed the emphasis on ensuring the model could spot 10 common surgical instruments.
• The tracking module monitors the movements of a scrub nurse’s hands. The basic assumption is as follows: when an item leaves the table, i.e., the nurse selects it from the instrument table upon the surgeon’s request, it is being used. Logically, if the item never leaves the table, it is safe to assume it stays unused, whether it is opened or not.
• The video classification module analyzes, tracks, and classifies activities contained in a video recorded during the surgery. By processing both image frames and spatiotemporal relationships between adjacent frames, it recognizes the actions performed by the scrub nurse (e.g., unpacking the surgical items, selecting and passing instruments to the surgeon, returning them to the table). Thereby, the module can predict the probability of surgical tools moving from and onto the instrument table.

Reaching >95% in Identification Quality and Tracking Reliability

The fully working PoC, ready for demo purposes, was delivered in just 3 months after the Customer’s initial request. The timing was crucial, as the Customer needed to demonstrate the concept to investors to get additional funding for further solution implementation. ScienceSoft achieved the maximum expected KPIs, namely 80% in identification quality and tracking reliability (used vs. unused tools). After 2 additional iterations, 3 months each, these metrics exceeded 95%.

Technologies and Tools

Python, Pytorch.