AI software could help retailers deter shoplifting by analyzing body language

It’s watching, and knows a crime is about to take place before it happens.

Vaak, a Japanese start-up, has developed artificial intelligence software that hunts for potential shoplifters, scanning footage from security cameras for fidgeting, restlessness and other potentially suspicious body language.

AI is usually envisioned in a smart personal assistant or self-driving car, but it turns out the technology is pretty good at spotting nefarious behavior too. Like in a scene from the movie “Minority Report,” algorithms analyze security-camera footage and alert staff about potential thieves via a smartphone app.

The goal is prevention: If the target is approached and asked if he or she needs help, there’s a good chance the theft never happens.

Vaak made headlines last year when it helped to nab a shoplifter at a convenience store in Yokohama. Vaak had set up its software in the shop as a test case, which picked up on previously undetected shoplifting activity. The perpetrator was arrested a few days later.

“I thought then, ‘Ah, at last!’” said Vaak founder Ryo Tanaka, 30. “We took an important step closer to a society where crime can be prevented with AI.”

Shoplifting cost the global retail industry about $34 billion in lost sales in 2017 — the biggest source of shrinkage, according to a report from Tyco Retail Solutions. That amounts to about 2% of revenue, which can make a huge difference in an industry known for razor-thin margins.

The opportunity is huge. Retailers are projected to invest $200 billion in new technology this year, according to Gartner Inc., as they become more open to embracing technology to meet consumer needs, as well as improve bottom lines.

“If we go into many retailers whether in the U.S. or U.K., there are very often going to be CCTV cameras or some form of cameras within the store operation,” said Thomas O’Connor, a retail analyst at Gartner. “That’s being leveraged by linking it to an analytics tool, which can then do the actual analysis in a more efficient and effective way.”

Because it involves security, retailers have asked AI software suppliers such as Vaak and London-based Third Eye not to disclose their use of the anti-shoplifting systems. It’s safe to assume, however, that several big-name store chains in Japan have deployed the technology in some form or another. Vaak has met with or been approached by the biggest publicly traded convenience-store and drugstore chains in Japan, Tanaka said.

Big retailers have already been adopting AI technology to help them do business. Apart from inventory management, delivery optimization and other enterprise needs, AI algorithms run customer support chat bots on websites. Image and video analysis is also being deployed, such as Amazon.com Inc.’s Echo Look, which gives users fashion advice.

“We’re still just discovering all the market potential,” Tanaka said. “We want to keep expanding the scope of the company.”

Founded in 2017, Vaak is testing in a few dozen stores in the Tokyo area. The company began selling a market-ready version of its shoplifting-detection software this month, and is aiming to be in 100,000 stores across Japan in three years. It has 50 million yen ($447,000) in funding from SoftBank Group Corp.’s AI fund and is in the middle of its series A round, seeking to raise 1 billion yen ($8.9 million).

What makes AI-based shoplifting detection a straightforward proposition is the fact that most of the hardware — security cameras — is usually already in place.

“Essentially this is using something that’s been underutilized for decades,” said Vera Merkatz, business development manager at Third Eye. Founded in 2016, the start-up offers services similar to Vaak’s in the U.K. market, where it has a deal with a major grocery chain. Third Eye is looking to expand into Europe.

The ability to detect and analyze unusual human behavior also has other applications. Vaak is developing a video-based self-checkout system and wants to use the videos to collect information on how consumers interact with items in the store to help shops display products more effectively.

Beyond retail, Tanaka envisions using the video software in public spaces and train platforms to detect suspicious behavior or suicide jumpers. At Third Eye, Merkatz said she’s been approached by security management companies looking to leverage their AI technology.

“The potential is broad since it can be applied outside of shoplifting prevention and outside of retail — such as with manufacturing or other types of marketing,” said Hiroaki Ando, a retail consultant at Ernst & Young Advisory & Consulting Co. in Tokyo.