Robert De Niro in a motion capture suit? Not going to happen. Enter new technology

Al Pacino and Robert De Niro emoting in a motion-capture suit with dozens of sensors attached to their heads? That’s not going to happen.

So when Martin Scorsese needed a work-around to conventional performance capture technology enabling him to make Pacino (as labor boss Jimmy Hoffa), Robert De Niro (hitman Frank Sheeran) and Joe Pesci (mob boss Russell Bufalino) appear younger in “The Irishman,” the director turned to Industrial Light & Magic visual effects supervisor Pablo Helman. On the set of “Silence” five years ago, Helman recalls, “Marty gave me the script for ‘The Irishman’ and said he’d been trying to make this movie with Bob De Niro for 10 years.”

The story spans five decades, and Scorsese’s septuagenarian star had become too old to portray the title character in his prime. “The performance capture thing everybody did at the time used markers and helmet cams and all this other stuff,” says two-time Oscar nominee Helman. “We agreed there was no way Bob De Niro would want to wear gray pajamas and perform in a mo-cap studio.”

Conveniently, Helman happened to be developing a stripped-down approach to performance capture. “Marty was very curious about it and asked me, ‘How would you do this [de-aging] for ‘The Irishman’? ‘“

The answer came a few months later after one false start. Helman and his team at ILM initially had a stand-in for De Niro reenact a scene from the 1990 crime classic “Goodfellas.” That didn’t work. “Marty and [editor] Thelma Shoemaker were so familiar with Bob De Niro’s presence, the way he walked — there wasn’t enough truth in it for them,” Helman says. So De Niro reenacted the scene himself, playing 47-year-old “Jimmy Conway” in the film’s classic Cadillac sequence.

Over the next 10 weeks, Helman rendered the actor as a 3-D computer model, then “re-targeted” the contours of his face drawing on reference images of a much younger De Niro. Helman went to New York and showed Scorsese his proof of concept.

“Marty was elated. It took the [de-aging] burden off the actors and put the burden on the cameras, using lighting and scientific triangulation to show where the character is in 3-D space. This meant the actors could be on set with no makeup, no markers, no restrictions about where they can move. Marty and his team completely got it and said ‘Go ahead, build it.’ ”

During the shoot, cinematographer Rodrigo Prieto followed the characters with a rig anchored by a director’s camera flanked by two “witness” cameras. On the first day of production in upstate New York, Helman remembers watching De Niro and Pesci in action.

“We’re at this gas station. ‘Bob, you’re late 30s; Joe, you’re early 50s, so you have to behave that way.’ They do the scene, and that’s it. We did our data gathering, collecting information about the light, because that’s how our system creates 3-D geometry. Then, we moved on to the next scene. We did that for 108 days.”

As analog production proceeded, the digital pipeline pumped furiously. Deploying a new software program called Flux, Helman and an army of some 500 VFX artists from San Francisco; Vancouver, Canada; England; China and India imported footage from the shoot to create digital doppelgangers of the actors, who were then “de-formed” into younger versions of themselves.

Drawing from a massive array of archival De Niro-as-a-younger-man images, they created multiple versions of Frank Sheeran as he matures through the story from late 20s to age 62. Pesci as Bufalino went back in time to age 53, then 62, then 70 before the actor transitioned into physical makeup. Pacino’s Jimmy Hoffa started at 44. “We were going to have him go into makeup at 60 because Hoffa dies at 62,” Helman says. “But Al didn’t want to do makeup so he stayed digital.”

Pacino’s fondness for digital de-aging came as no surprise to Helman.

“It’s so much easier for the actors,” he says. “We never used keyframe animation to change the movement of an eyebrow or anything because, in our digital world, the difference between a smile and a wince might only be two pixels. Marty loved the performances so much, it was really important that we not change the actors’ choices with animation. We just made them younger, that’s all.”