The TraumaPod system consists of a teleoperated surgical robot remotely controlled by a surgeon from a central base, along with several autonomous robotic systems providing support for the procedure by delivering surgical supplies and changing the tools in use by the surgical robot. The system is expected to be deployed inside unmanned Armed Forces vehicles used in far-forward battlefield environments.

RST has developed a machine vision system, called the MVS, for the TraumaPod, that tracks and counts the surgical supplies and tools using an array of cameras mounted throughout the Pod. This capability is important for several reasons: it creates an automatic visual record of every supply and tool that was used during an operation. By counting every supply that enters the surgical field, it improves safety in the fast-paced and frequently chaotic environment of trauma surgery. And the MVS brings intelligence and situational awareness to the autonomous systems in the Pod, ensuring that the surgeon will not be distracted by the mistaken delivery of empty or disorganized trays of supplies. The software behind this system is a customized version of the same advanced machine vision software used in Penelope.

Counting Threads

The primary job of the MVS is to count surgical supplies in trays that are held by the grippers of autonomous robotic systems. The counts are performed each time the tray is delivered to the surgeon. The surgeon may have used zero, one or more of the supplies. If the tray is empty, the system disposes of it.

As seen in the photo above, the types of supplies held in these trays can include (clockwise from top left) sutures, sponges, shunt kits with clamps, and surgical peanuts (small gauze bundles). When a count is requested, the MVS takes a picture and analyzes it to find the supply tray. It then places a set of masks (gray trapezoids in the image at left) over the spots in the tray where it expects the supplies to appear. Each mask is checked for the presence or absence of a supply. The software highlights the pixels comprising each detected supply in a different color and reports the count back to the system.

The MVS uses dynamic masking to enable correct supply counts regardless of how the tray appears in the image. It can handle changes in angle, position and distance from the camera.

Future MVS capabilities are currently under development. These include tracking the tools used by the surgical robot, and visual servoing to detect any misalignment between the interacting robotic systems before a collision can occur.