The SolarWinds hacking campaign has raised a lot of questions about why intruders behind the operation, believed to be from Russia, weren’t caught sooner as they moved through government and private networks for months undetected.
Some have wondered why the National Security Agency’s signals intelligence and skilled hackers, positioned in adversary networks around the world, failed to learn of the attack while it was being planned or in progress, in the way that Dutch spies monitoring Russian state hackers in 2016 discovered that they had hacked the Democratic National Committee and notified the FBI.
Others have faulted the government’s multi-billion-dollar Einstein system for failing to detect the hackers once they were inside government networks. Einstein, a Department of Homeland Security system, uses sensors that sit outside civilian government networks to scan incoming and outgoing traffic for known malicious code or activity, such as phishing attempts, communication with hacker command-and-control servers or exfiltration of stolen data.
Lawmakers and others have also used the hacking campaign to raise questions about restrictions against the NSA conducting domestic surveillance. The NSA has no authority to monitor civilian private networks inside the U.S. for malicious activity — only military networks — leading some to suggest this created a blind spot for the nation’s most technically skilled intelligence agency from helping to detect the attacks.
The hackers used U.S.-based servers operated by Amazon Web Services (AWS) and other providers to communicate with the malicious backdoor they installed on victim networks.
Lawmakers at a Senate Intelligence hearing in February suggested the hackers intentionally leveraged these domestic systems to exploit the NSA’s blind spot. Anne Neuberger, deputy national security advisor to the president and former head of the NSA’s Cybersecurity Directorate, also seemed to underscore this during a press conference about the campaign when she noted that “the intelligence community largely has no visibility into private sector networks.”
And retired Adm. Mike Rogers, former director of the NSA, said last week during a panel discussion hosted by the Kellogg School of Management, at Northwestern University, said, “You can’t defend something you can’t see.”
But experts say that many assumptions and assertions about the SolarWinds campaign are false.
“There is no actual technical, practical ability for NSA-level surveillance to have detected either the SolarWinds or the Microsoft Exchange attacks,” says Katie Moussouris.
The use of U.S. infrastructure in hacking campaigns is not new or a tactic unique to the SolarWinds campaign. Nation-state hackers and cybercriminals have long used U.S.-based computers and servers as proxies through which to route attacks or as their command-and-control infrastructure. In some cases the hackers hijack vulnerable servers; other times they rent servers in data centers, like other customers, though using stolen credit card numbers and IDs to do so. In 2019, according to a report by Spamhaus Malware Labs, more than 4,000 servers controlling malicious botnets were based in the U.S. Even the use of AWS servers specifically is not new. In 2009, security researchers found U.S.-based Amazon server running the Zeus botnet.
“This is more than 25 years of standard-operating-procedure for attackers. There is nothing new here,” says Katie Moussouris, founder and CEO of Luta Security.
Suggestions that the campaign might have been caught if the NSA or DHS were allowed to monitor U.S. private networks is also untrue, she said.
“There is no actual technical, practical ability for NSA-level surveillance to have detected either the SolarWinds or the Microsoft Exchange attacks,” she said, referencing a second widespread nation-state hacking operation, believed to be launched by China, that targeted Microsoft Exchange servers. Moussouris points to the fact that both the Department of Defense and more than half a dozen civilian government agencies were victims of the campaigns as evidence that monitoring of private domestic networks would not have caught the hackers sooner.
“The NSA capabilities failed to detect it in government systems where they’re supposed to be looking,” she noted. “All of these arguments are transparently trying to push an agenda of expanding domestic surveillance, and they don’t stand up to technical scrutiny… There is absolutely nothing that we currently have that could have detected any of these things.”
Network monitoring uses signatures and heuristics — designed to find behavioral anomalies on a network — to detect malicious code and suspicious activity. Signatures are only useful, however, if the attacker is using code that is the same or similar to known malicious code used in previous attacks. It won’t necessarily detect malware used for the first time. As for anomalous behavior, the hackers behind the SolarWinds campaign took steps to make sure their activity on victim networks looked normal.
They did this in part by stealing the credentials of network administrators, making it look like their activity was legitimate. The email they siphoned from victim networks was stolen in small increments and encrypted to bypass monitoring systems. Because it was transmitted to servers based in the U.S., it did not raise automated alarms from network detection systems that are configured to alert on traffic going to suspicious domains. It was only after one of the hackers acted recklessly in a way that triggered an alert inside the network of the security firm FireEye that they were caught. But by then the intruders had already been in FireEye’s network undetected for months, according to Senate testimony by Kevin Mandia, FireEye CEO, last month.
Asked by Sen. Roy Blunt (R- Missouri) how long the intruders had been in FireEye’s network before they were caught, Mandia replied, “a couple of months from initial access, but the attacker wasn’t alive every single day… In other words, they were on our system for maybe three hours on one day, a week would go by, a couple of hours on another day. We weren’t a full-time job for the intruders that broke into us because they had broken into 60-plus other organizations, if not a hundred…. There’s several days of activity before we detected them. But over time, it was several months [that they were in the network].”
Blunt noted that because FireEye is in the business of detecting intruders, its experts were in a better position than anyone else to notice the hackers in their network but still missed them until they made a mistake.
“My fear is that we will in fact legislate in panic as we always do, and we end up with the Patriot Act of cyber where we’re freaking out. I think you’d be crazy not to worry about rapid legislation in this space right now.”
- Dave Aitel
Dave Aitel, founder and former CEO of the security firm Immunity as well as a former research analyst at the NSA, says that talk about increasing surveillance to catch nation-state attacks is misguided.
“Even If you had the world’s best surveillance, would it have detected SolarWinds? The answer is no,” he said. “But it would have detected other things,” such as known threats.
Introducing government surveillance of private networks, however, raises a host of new problems. Aside from the fact that the private sector and their domestic and international customers do not welcome increased U.S. government monitoring of networks and traffic — Microsoft President Brad Smith famously called the U.S. government an “advanced persistent threat” following revelations about NSA spying on internet traffic in 2013 — the problem with increasing network monitoring on a mass scale is the inability to analyze it all, Aitel says. The amount of data to be monitored would be immense and impractical.
“We are worse at analytics than we are at our visibility,” he says. “And once you have the monitoring in place, your adversary will adjust [to thwart the monitoring]. In places where they aren’t using encryption, they will use encryption” for example.
Senior Biden administration officials recently told the New York Times that while the White House is weighing options for addressing the hacking campaign, it has no plan to push Congress to grant intelligence agencies power to monitor networks inside the U.S.
But Aitel says that other options being proffered to address the problem have their own issues as well.
He points to an executive order signed by former President Trump in January — one of the last acts of his administration — that is aimed at preventing foreign adversaries and criminal hackers from using U.S. cloud infrastructure to conduct malicious operations. Cloud and other service providers would have to verify the identity of foreign customers, maintain records of their transactions. The rules established under the order could also require providers to monitor for malicious use of their infrastructure and take action to close it down when detected.
The order, however, has been criticized for failing to understand the onerous impact this will have on businesses. It also won’t prevent malicious attacks; it will simply drive hackers to providers outside the U.S. or to domestic providers who fail to follow through on the requirements.
“My fear is that we will in fact legislate in panic as we always do, and we end up with the Patriot Act of cyber where we’re freaking out,” Aitel says. “I think you’d be crazy not to worry about rapid legislation in this space right now.”
He says the real blind spot is the lack of foresight into where nation-state attacks will go next.
“It’s not just what are they doing now but what will they do in reaction to what we will do. Because they're playing chess not checkers.”