Putting More Reality Into Virtual Reality

BERKELEY — In 1991, Paul Debevec daydreamed of flying cars.

The dreams were inspired by “Back to the Future” and “Star Wars,” two of Debevec’s favorite movies. Unfortunately, getting Debevec’s own little gray Chevette into the air seemed like a problem too difficult even for a resourceful UC Berkeley computer science student.

So Debevec did the next best thing.

“I made a 3-D model of the car, but I did it from photos, so instead of looking like a computer-generated car, it looked a lot like my Chevette,” Debevec says. “It even had all the spots where the paint was missing. Then I made it fly.”

Seven years later, Debevec, now 25, and his colleagues crowd around a video monitor in a Berkeley computer lab. All eyes are fixed on the screen, showing a video that was completed hours ago--just in time for submission to SIGGRAPH 97, the annual computer graphics conference to be held in Los Angeles this summer.

The video clip depicts Debevec placing a small architect’s model of Berkeley’s Sather bell tower in the center of the campus. The camera zooms in on the model, sweeps around it at breakneck speed, and pulls back out again--magically revealing the entire Berkeley campus below the tower. The lab erupts with excited laughter and applause.

“This video is playing off the idea of what’s a model and what’s real,” Debevec explains as his friends watch the tape again and again. “We’ve got the real thing, a physical architect’s model, and the computer model. The film fades seamlessly between these in every possible way.”

Hollywood is already hot on his system of modeling and rendering computer-generated architecture from still photographs. In November, he was flown to Warner Digital Effects in Hollywood, where he applied some of the same techniques he first developed making his car fly to aid in increasing the realism of Gotham City for this summer’s “Batman and Robin.”

Between interviews for professorial positions, Debevec has also been taking meetings at Industrial Light & Magic, the George Lucas-owned special-effects powerhouse.

“Paul has definitely made some innovative inroads into important technology,” says H.B. Siegel, chief technology officer at ILM. “His system will offer a lot of versatility to people in the filmmaking businesses.”

Debevec hopes to be using his techniques soon to create fully interactive, photo-realistic virtual worlds, in which users can explore the space in real time.

At best, virtual worlds today have cartoon scenery of the type appearing in the movie “Toy Story.” Even photography-based virtual reality available on CD-ROM or the Internet--such as Apple’s QuicktimeVR, Black Diamond’s Surround Video software, and RealVR Traveler from Live Picture & RealSpace--only allow the user to view a panorama from a fixed point, not wander through it.

But once Debevec combines interactivity with the models he creates with Facade, virtual reality will seem much more real. To make his video, Debevec started with 16 standard photographs of the Sather tower and surrounding area, including four taken from an inexpensive kite camera. Debevec used his Facade software to model the scene by matching the edges of basic geometric blocks, such as boxes, to outlined segments in the photograph.

The software then calculates the shape and position of each block and snaps the model into alignment with the photographs, a technique called photogrammetric modeling. Finally, the original photographic images are projected onto the reconstructed geometry, making the scene photo-realistic.

“We’re getting the geometry and the appearance directly from photographs. That way you get a look that’s very much like the real world,” he says. “The attractive thing is that as soon as we have virtual reality technology that lets you see high-resolution models in real time, you’ll actually be able to interact with the film.”

Previously, to create a 3-D model of an object like a building, studios would need to obtain architectural plans, “or go out with rulers,” Debevec says. Then the textures, the faces of the objects, were often hand-painted in, yielding results that “don’t look particularly real,” he says.

Not only did Debevec model 40 photo-realistic buildings in about a week, but no helicopter camera could have captured footage as closely and steadily as his virtual camera did.

“Once people realize this technology exists, they may be able to do the films they didn’t know how to create the effects for,” Debevec says.

For more information, go to https://www.cs.berkeley.edu/~debevec

*

David Pescovitz (pesco@well.com) is the coauthor of “Reality Check” (HardWired, 1996) and a contributing editor at Wired magazine.