This Bot hunts software bugs for the Pentagon

At the end of last year, David Haynes, a security engineer at internet infrastructure company Cloudflare, found himself staring at a strange picture. “It was pure gibberish,” he says. “A whole bunch of gray and black pixels, made by a machine.” He declined to share the image, saying it would be a security risk.

Haynes’ caution was understandable. The image was created by a tool called Mayhem that probes software for unknown security vulnerabilities, created by a startup from Carnegie Mellon University called ForAllSecure. Haynes had tested him on Cloudflare software that resizes images to speed up websites, and provided him with several sample photos. Mayhem turned them into cursed, cursed images that crashed photo processing software by triggering an unnoticed bug, a weakness that could have caused headaches for customers paying Cloudflare to keep their websites functioning properly.

Cloudflare has since made Mayhem a standard part of its security tools. The US Air Force, Navy, and Army have also used it. Last month, the Pentagon awarded ForAllSecure a $ 45 million contract to expand the use of Mayhem in the U.S. military. The department has a lot of bugs to find. A 2018 government report found that almost all weapon systems tested by the Defense Ministry between 2012 and 2017 had serious software vulnerabilities.

Mayhem is not sophisticated enough to completely replace the work of human bug seekers, who use their knowledge of software design, code reading skills, creativity, and intuition to find faults. But ForAllSecure co-founder and CEO David Brumley says the tool can help human experts do more. The world’s software has more security holes than experts have time to find, and more and more are coming every minute. “Safety isn’t about safety or insecurity, it’s about how fast you can move,” says Brumley.

Mayhem was born from an unusual hacking contest in 2016 at a Las Vegas casino ballroom. Hundreds of people showed up to watch the Cyber ​​Grand Challenge, hosted by Pentagon research agency Darpa. But there were no humans on stage, only seven well-lit computer servers. Each hosted a bot that attempted to find and exploit bugs on other servers, while also finding and fixing its own flaws. After eight hours, Mayhem, made by a team from Brumley’s Carnegie Mellon Safety Lab, took home the top prize of $ 2 million. His magenta-lit server landed in the Smithsonian.

Brumley, who is still a professor at Carnegie Mellon, says the experience convinced him that setting up his lab could be useful in the real world. He put aside the offensive abilities of his team’s bot, defensive reasoning was more important, and set about marketing it. “The Cyber ​​Grand Challenge has shown that fully autonomous security is possible,” he says. “Computers can do a reasonably good job. “

The governments of China and Israel thought so too. The two offered contracts, but ForAllSecure signed with Uncle Sam. He got a contract with the Defense Innovation Unit, a Pentagon group trying to quickly introduce new technology into the US military.

ForAllSecure was challenged to prove Mayhem’s courage by looking for flaws in the control software of a commercial airliner with a military variant used by US forces. Within minutes, the automatic hacker discovered a vulnerability which was then verified and fixed by the aircraft manufacturer.

Other bugs discovered by Mayhem include the one discovered earlier this year in OpenWRT software used in millions of network devices. Last fall, two interns at the company got payout from Netflix’s bug bounty program after using Mayhem to find a loophole in software that allows people to send videos from their phones to a TV.

Brumley says interest from auto and aerospace companies is particularly strong. Cars and airplanes are increasingly dependent on software, which has to work reliably for years to come and is rarely, if at all, updated.