Generative AI is already ubiquitous in almost every industry, whether we like it or not, and cybersecurity is no exception. The possibility of AI-accelerated malware development and autonomous attacks should alarm any system administrator even at this early stage. Wraithwatch is a new security team that aims to fight fire with fire, deploying good AI to combat the bad guys.
The image of righteous AI agents fighting evil ones in cyberspace is already pretty romanticized, so let’s make it clear from the start that this isn’t Matrix-style hand-to-hand combat. It’s about software automation that enables malicious actors the same way it does the rest of us.
Employees of SpaceX and Anduril until just a few months ago, Nik Seetharaman, Grace Clemente and Carlos Más witnessed firsthand the storm of threats to which every company with something valuable to hide is subject at all hours (think of the aerospace industry , defense, finance).
“This has been going on for more than 30 years, and LLMs are only going to make things worse,” Seetharaman said. “There’s not enough dialogue about the implications of generative AI on the offensive side of the picture.”
A simple version of the threat model is a variation of a normal software development process. A developer working on a typical project could make some of the code personally and then tell an AI co-pilot to use that code as a guide to create a similar feature in five other languages. And if it doesn’t work, the system can iterate until it does, or even create variants to see if one works better or is easier to audit. Useful, but not a miracle. Someone is still responsible for that code.
But think about a malware developer. They can use the same process to create multiple versions of malware in just a few minutes, protecting them from “fragile” surface-level detection methods that look for package sizes, common libraries, and other telltale signs of malware. or its creator.
“It’s trivial for a foreign power to point a worm at an LLM and say ‘hey, mutate into a thousand versions’ and then release all 1,000 at once. In our testing, there are uncensored open source models that are happy to take your malware and mutate it in any direction you want,” Seetharaman explained. “The bad guys are out there and they don’t care about alignment; you yourself have to force the LLMs to explore the dark side and map them to know how you will really defend if that happens.”
A reactive industry
The platform that Wraithwatch is building, and hopes to have commercially operational next year, has more in common with wargaming than with traditional cybersecurity operations, which tend to be “fundamentally reactive” to threats that others have detected, they said. . The speed and variety of attacks could soon overwhelm the largely manual and human-driven cybersecurity response policies that most companies use.
As the company writes in a blog post:
New vulnerabilities and attack techniques (occurring weekly) are difficult to understand and mitigate, requiring in-depth analysis to understand the underlying attack mechanics and manually translate that understanding into appropriate defensive strategies.
“Part of the challenge for cyber teams is that we wake up in the morning and learn about a day zero. [the name given to security vulnerabilities where the vendor has no advance notice to fix them] — but when we read about it, there are already blogs about the new variation it has mutated into,” Clemente said. “And if you’re at SpaceX or Anduril or the US government, you’ll get a new custom version made just for you. “We can’t rely on waiting until they hit someone else.”
Although these custom attacks are now largely human-made, as are the defenses against them, we’ve already seen the beginnings of generative cyber threats in things like WormGPT. This may have been rudimentary, but it is a question of when, not if, improved models will be used to solve the problem.
Más noted that current LLMs have limitations in their capabilities and alignment. But security researchers have already demonstrated how major code generation APIs, such as OpenAI, can be tricked into helping a malicious actor, as well as the aforementioned open models that can run without alignment restrictions (bypassing ” Sorry, I can’t create malware.”) ”-type answers).
“If you start being creative in how you use an API, you can get a response that maybe you weren’t expecting,” Más said. But it’s about more than just coding. “One of the ways that agencies detect or suspect who is behind an attack is that they have signatures: the attacks they use, the binaries they use… imagine a world where you can have an LLM generate signatures like that. You click on a bot and you have a new APT [advanced persistent threat, e.g. a state-sponsored hacking outfit].”
It’s even possible, Seetharaman said, that new agent-like AIs trained to interact with multiple software platforms and APIs as if they were human users could be activated to act as semi-autonomous threats to attack in a persistent and coordinated manner. If his cybersecurity team is prepared to counter this level of constant attack, it will likely be only a matter of time before a breach occurs.
War games
So what is the solution? Basically, a cybersecurity platform that leverages AI to adapt its detection and countermeasures to what an offensive AI is likely to throw at you.
“We were very deliberate about being a security company that does AI, and not an AI company that does security. We have been on the other side of the keyboard and we have seen until the last days [at their respective companies] the type of attacks they launched at us. We know how far they will go,” Clemente said.
From left to right, Wraithwatch co-founders Carlos Más, Nik Seetharaman and Grace Clemente. Image credits: Specter Surveillance
And while a company like Meta or SpaceX may have top-notch security experts in-house, not every company can support a team like that (think a 10-person subcontractor for an aerospace company), and in any case, the tools they use. working with him might not be up to the task. The entire reporting, response and disclosure system can be challenged by malicious actors authorized by LLMs.
“We’ve seen every cybersecurity tool on the planet and they’re all lacking in some way. We want to sit as a command and control layer over those tools, tie a thread through them and transform what needs transformation,” Seetharaman said.
By using the same methods that attackers would use in an isolated environment, Wraithwatch can characterize and predict the types of variations and attacks that LLM-infused malware could deploy, or so they hope. The ability of AI models to detect signals in noise is potentially useful for establishing layers of perception and autonomy that can detect and possibly even respond to threats without human intervention; I don’t want to say that everything is automated, but the system could be prepared to block a hundred probable variants of a new attack, for example, as soon as its administrators want to use up patches for the original.
“The vision is that there is a world where when you wake up you wonder if you’ve been raped yet, but Wraithwatch is already simulating these attacks by the thousands and saying Here are the changes you need to make.and automate those changes as much as possible,” said Clemente.
Although the small team has “several thousand lines of code” in the project, it’s still early days. Part of the argument, however, is that, as certain as malicious actors are exploring this technology, large corporations and nation-states are likely to be too; or at least, it is healthy to assume this rather than the opposite. A small, agile startup made up of veterans of seriously threatened companies and armed with a ton of venture capital money could well outperform the competition, unencumbered by the usual corporate baggage.
The $8 million seed round was led by Founders Fund, with participation from XYZ Capital and Human Capital. The goal is to get it up and running as quickly as possible, since at this point it’s fair to consider it a race. “Since we are coming from companies with aggressive timelines, the goal is to have a robust MVP with most of the features implemented for our design partners in the first quarter of next year,” with a broader commercial product arriving in late 2024 Seetharaman said.
It may all seem a bit far-fetched, talk of artificial intelligence agents laying siege to American secrets in a secret war in cyberspace, and we’re still a long way from that particular airport thriller propaganda. But an ounce of preparation is so worth it, especially when things are as unpredictable and fast as they are in the world of AI. Hopefully the problems that Wraithwatch and others are warning about are at least a few years away, but in the meantime, it’s clear that investors believe that those with secrets to protect will want to take preventive measures.
