3D Perception in Animation

I recently had the chance to talk to a friend about her stereoscopic work in the pharmaceutical industry. This relates to Amanda's post on the NASA 3D Sun Project in that it is important to remember there are applications for this technology outside of the entertainment industry.

My friend used sterescopics to help scientists visualize proteins. There are three ways to represent proteins: primary (the written scientific formula), secondary (a diagrammed 2D representation), and tertiary (the addition of Z space, a 3D model). It is obviously in tertiary representation that stereoscopics can enhance viewing. Consider for example this image:

It is hard to determine depth even though it is a 3D representation. It reminds me of an example Perry Hoberman had in his seminar presentation to us. He displayed a confusing image composed of a series of lines, but when viewed through our red and cyan filters the relationship of the lines to each other in Z space became much clearer. It is easy to imagine how this could benefit scientists contemplating molecules radically more complex than this one. 3D visualization is also valuable in that it allows them to run virtual tests and simulations. Contemplating all of this made me very aware of how processing power can improve quality of life, not just decrease rendering times. Think about how much more accurately and faster scientists can research drugs with this technology at their fingertips, technology that may have been unavailable just a few short years ago.

These resources aren't stereoscopic, but if you're interested in viewing some 3D molecular models check out this site or download RasMol, A molecular graphics visualization tool that comes with a couple of sample files. There are many ways to view the same molecule, here's an example:

Finally, from an industry point-of-view it's interesting that stereoscopic technology leaders in science and entertainment are now under one roof once StereoGraphics became a part of RealD.

- John (with special thanks to Heather Barbre)

I had the opportunity to ask a recent graduate of USC's animation program, who has worked in stereoscopy, some questions about what working on stereo features and in the industry is really like. Interview as follows!

How did you get involved with working on the 3D Beowulf project? Was stereoscopy something you were interested in as a student?

Stereoscopic 3-D wasn’t my interest at all – in fact, I was recommended as a lighter by a lead lighter at Sony and I guess at the time they needed stereo people more. So they were only wanting to hire me for that. In many ways, doing stereo at Sony is similar to being a lighter, except you have to know the lighting package (Katana) and compositing package (Bonsai) and then also this other layer of technical things that lighters don’t really deal with.

When a supervisor at Sony sees your reel and decides they might be interested, they give your number to a recruiter. This person knows very little about actual production; his only job is to get people hired. I’ve dealt with a few recruiters, some are laid back and some are fast talker types. One tip I have is that if you decide to turn down a job, be aware that the recruiter is not emotionally invested in this process and won’t take offense. Be sure to tell them “but I may be available on these days and am interested in these kinds of work, please keep me on your list.”

Can you describe more about what this process entailed?

3-D is the very very final stage of a shot’s creation, we get handed the finaled shot from the lighting department. Our department grew in size as the show was finishing, it was in the teens when I started but at the end they even had some lighters working with us and we were about 45 strong.

Very little of stereoscopic work is creative, and if it is, then you’re coming up with creative workarounds for technical problems. Making a film stereoscopic essentially means creating a right-eye image to correspond to the left-eye image that the lighting department has made. Sometimes this means completely rerendering a shot, but Sony has some shortcuts (that sometimes work) which can be faster than rerendering. There’s always a balancing act between artist hours and render hours.

Sony’s lighting department operates very differently from Pixar’s, they are somewhat opposite in approach. Pixar does almost no compositing, they actually light the entire scene, shot by shot, and render the whole thing out, lens effects and all. Sony usually separates character passes from background passes, etc., and more often than not they will break down passes for diffuse, specular, indirect, reflection, and so on.

Some of the comps are very complicated, and often the stereoscopic artist is handed these an insane comp and has to spend hours just figuring out what the hell is going on in it. If you want to make friends in the industry, teach yourself now how to properly organize and label your work so that anyone could pick it up and know what’s going on. Scripts should flow left->right and top->bottom, nodes should be named logically and shouldn’t be sitting on top of each other.

The problems we ran into in our department were too many to count, every new shot was like a new set of problems you’d never dealt with. Sometimes things don’t work right and you have to do roto fixes. Sometimes you try to rerender but the original lighting setup has been corrupted and you have to tweak about a thousand different controls to figure out what’s wrong.

What type of skills did you need to use for this job? Were they related to your interests or training in animation? Do you think you’d want to work on a similar type of project again? What did you learn from it? What did you like the best, the worst?

Most of what I got out of working at Sony is learning what a big studio is like, learning their pipeline and efficient Linux techniques and some basic fundamental stuff. In order to do well in this job, you’d definitely need a lot of common sense and patience, at least mid-level comfort with node-based compositing, and some rendering fundamentals. I would strongly recommend taking notes and figuring out which supervisors and coworkers know the most so you can ask them questions when needed.

I’m not interested in working stereoscopic again. It was a good foot-in-the-door experience, but I found it creatively unfulfilling and my career goals don’t really align with it. I met a couple people at Sony I liked, and I got a better sense of what the industry is about. But I think it’s important to stay focused on getting to where you want to be, wherever that is.

Would you or have you ever considered outputting any of your own creative work in a 3D format? If someone is considering finishing a project in 3D, what types of issues should they address from the onset when they are making a film?

I hadn’t really thought of making a 3-D film, as you can be very limited by where it can be screened (finding polarized projectors is no small task). I think if I chose too, I’d take the economical route and make it pseudo-3-D. In other words, use a sort of virtual multi-plane system so that the eyes perceive the background as being far away, the characters as closer, but these different planes of dimension don’t have true depth. This way, you get a sense of space but you can just offset the different elements in your composite instead of actually rerendering the right eye. But this may just be my laziness talking. I personally feel like 3-D is the re-awakening of a fad and the benefits can be questionable and will fade with time.

Another issue is that nobody really knows how to direct a film in 3-D. Things we use in regular film, like depth of field, don’t work as well in 3-D and so suddenly you’re tasked with leading the audience’s eye without techniques you might have normally used. You have to think about things like shot continuity in 3-D, like if you do a 3-D jump cut, your audience’s eyes will cross and they’ll feel even more nauseous than before. While someone might feel excited about pioneering direction in this medium, I personally just don’t feel like the thrill of 2-D filmmaking is exhausted, so I’m not so interested.

Maybe the biggest issue to address is, does the film I’m envisioning actually gain anything from being in 3-D? The parts of Beowulf where the 3-D really works are action scenes, I think, like when dragons are flying around. But there are some scenes where the 3-D camera is trying to make a shot ACTIONEY and it’s just silly. “Oh boy, the camera is moving along with the coin as it rolls along the ground!” It’s showy, it’s not really helping in telling the story. It takes you out of the film. As always, the important thing is to tell your audience a story and keep them interested in it.

-Amanda (Interviewee to remain anonymous!)