3D Artist Steve Rowse on “Designing the Original Robotic Surgery Simulator”

This year’s  Association of Medical Illustrators (AMI) Annual Conference brought together 400+ visual communication experts to share tales of cutting-edge medical procedures, newly discovered molecular processes, revealing scientific mysteries, and inspirational journeys that fascinate all of us. Mimic was honored to have three of our talented team members (Steve Rowse, Gordon Nealy and Emily Shaw) speak at the event, and even go on to win the prestigious “Charlotte Holt Award of Excellence” for the dV-Trainer, Mimic’s Robotic Surgery Simulator.

The following is a transcript from Mimic’s lead 3D Artist, Steve Rowse, who spoke about  “Designing the Original Robotic Surgery Simulator” at AMI 2014. In his speech, he talks about the role of a 3D medical robotic simulation artist and why he passed up a “dream job” as the Lead Technical Environment Artist on Microsoft’s Halo team to work with  Mimic Technologies.

“I’m a 3D medical robotic simulation artist. Of course, when I tell people this I typically get a blank stare followed by… ‘wow.. that’s umm, interesting.’ So what do I do?

I am responsible for the 3D content in our simulator software including soft and hard modeling, animation, lighting, rigging, rendering, FX, and the technical art pipeline.

I’d like to share a bit of the inner workings of our software and how we are helping to bridge the gap between medical illustration and real-time interaction through Virtual Reality, Simulation and Augmentation.

In previous jobs, I have created all kinds of content including, 2d and 3d animations, 2d and 3d environments, Vehicles and Weapons for Sony PlayStation SOCOM series, X-Box’s Need for Speed and children’s PC games.

In 2012 I passed up an offer for my “dream job” as the Lead Technical Environment Artist on Microsoft’s Halo team and took a position with Mimic because I became totally intrigued with what they were doing and recognized it as the next big idea – not just in 3D but in technology. Sitting on the corner of Simulation, Robotics, Virtual Reality and of course Medical illustration, I knew I just had to become a part of this. I saw clearly after interviewing with Mimic, that medical simulation is here and that the fusion of video games and Medical Illustration is inevitable. I believe that both careers are very important to our futures and we as professionals have to be aware of the other.

In other words it’s going to take a bridge to combine the two. Simulation is for the purpose of education and not the purpose of entertainment. So In recognition of this, when I was faced with my first major task at Mimic of redesigning and creating a look and feel for our exercises, I had to keep this in mind. My first reaction was to go photo-realistic. That not only presented a lot of technical and practical problems but my worry was that surgeons wouldn’t accept the mostly-realistic experience because they get caught up in the details of it not being exact. I came face to face to the Uncanny Valley. – the creepy experience we get when things are almost reality – but not quite.

The uncanny valley is a hypothesis in the field of human aesthetics which holds that when human features look and move almost, but not exactly, like natural human beings, it causes a response of revulsion among some human observers. The “valley” refers to the dip in a graph of the comfort level of humans as subjects move toward a healthy, natural human likeness described in a function of a subject’s aesthetic acceptability.

My choice then became to make as immersive experience as possible, but with a distinct message that this is not real thus not interfering with the users experience. This launched me into a Medical Illustration type position and not just a medical rendering position. My choice was to follow Pixar’s example of creating an experience rather than duplicating reality.  I believe that Illustration holds an invaluable contribution to the world of simulation, because after all aren’t we just bringing a text book to life by learning in a more experiential way? No matter what the technical advances are in CG, Artists and Illustration will always be an important and crucial piece of creating experiences and to enter this forum we must be both illustrators and game developers – a pretty high demand. In order to do this we use hand painted textures and simplified geometry. This also works to our advantage when is comes to more Software development issues like tiling textures and real time deformation.

So what else does the future hold in simulation?

One of the area’s that we are now working in is Augmented Reality using real video as a backdrop to answer questions in an interactive way about a surgery. Because this is actual video we avoid the uncanny valley, but we also lose our total interactive experience. In addition we are inventing new way to develop software in perceived 3D. What this means is that our development space is in 3D – not just our end product. While this is not artistically exciting at this point – what’s exciting about this is the technology behind it that will, I believe, be the next bridge to a hybrid of illustration, CT scanned 3D data and real time simulation which will give us the ability to offer virtual surgeries of real patients before the actual surgery is performed.

The thought I most want to share is simple and straightforward. Despite the ever present force of the technology push, and the desire for more “realistic looks”, illustration and art are not going to be replaced – they will just look slightly different. Illustration, for the purpose of education and communication will be essential in Virtual Reality and Simulation.”