In the wake of 9/11, U.S. intelligence agencies were granted vast surveillance powers, but those didn't stop Syed Rizwan Farook and Tashfeen Malik from committing the deadliest terror attack on U.S. soil since 2001. Farook and Malik, who killed 14 people and injured 21 others in San Bernardino, California, last week, appear to be exactly the kind of terrorists law enforcement is most afraid of in that they'd had no previous contact with authorities, did not communicate with known radicals, and appear to have hatched their plan here in the U.S.

Those are key reasons the two managed to evade watchdogs from the FBI, the U.S. National Security Agency, as well as local law enforcement. Measures like National Security Letters, which allow the FBI to obtain business records from banks, communication companies and other businesses, the No-Fly list, meant to prevent suspected terrorists from boarding commercial aircraft, Stingray cell phone surveillance, which tracks individual cell phones while collecting data on thousands of others, and even NSA metadata collection were primarily designed to collect intelligence that can stop foreign plots.

That’s not an encouraging reminder for police trying to stop suspects who aren’t engaged in an ongoing pattern of international communication or large money transfers, two of the more traditional methods of detection.

“We’d love to prevent all of these but it’s just not realistic,” Michael Chertoff, former Secretary of the Department of Homeland Security and co-author of the USA Patriot Act who now heads up his own consultancy group, said. “What you’re trying to do is look for relationships and connections that raise a red flag, which then may or may not prompt a closer look. The challenge is fundamentally collecting the haystack and giving yourself the ability to find the needle.”

Finding the needle becomes significantly more challenging when the suspects have had no prior contact with the police, are not on any FBI watch lists and purchased their weapons legally. Under the USA Freedom Act, enacted earlier this year, the U.S. National Security Agency is required to obtain a warrant to collect telephone metadata (call duration, numbers dialed, phone numbers involved and other data not including the contents of the phone call) from U.S. telecommunication companies. The NSA requests a warrant from the Foreign Intelligence Surveillance Court, which oversees the surveillance process regarding suspected foreign threats inside the U.S. and keeps all decisions classified, and serves it on a phone company, a process that might take days.

The government can still access Americans' phone data, but the process takes longer and there's no obvious way to use it to prevent cases like the San Bernardino shooting. 

“We’ve now moved to a model where the metadata is being held by telecom companies, and whether the U.S. government has the ability to detect those threats, to be honest, is still an open question,” Chertoff said. “The perennial problem we deal with is when there’s one or two people planning something in the privacy of their home and not a lot of interaction overseas and exchanging large sums of money. They’re not tripping a lot of wires.”

Farook corresponded by phone and social media with at least one person suspected of international terrorism, police told CNN. But FBI Director James Comey urged Americans “not to make too much” of reports that the assailants communicated with terrorists, adding that Farook was not under FBI surveillance. Even if he was, experts say, it’s unclear how police would have knowledge about his plot, or how they could have responded in real time.

GettyImages-499765032 U.S. President Barack Obama delivers a statement to the news media after receiving a briefing from his national security team, including FBI Director James Comey (right) and others, in the Oval Office at the White House Dec. 3, 2015, in Washington, D.C. Photo: Chip Somodevilla/Getty Images

The FBI failed to prevent a single terror attack based on the phone records collection program authorized as part of Section 215 of the Patriot Act, which preceded the USA Freedom Act, according to an FBI inspector general report examining the program between 2004 and 2009. The FBI has taken credit for stopping dozens of terrorist-related plots since 2001, including more than 10 arrests that could be traced to the Islamic State terrorist group before July 4, 2015. 

Part of the reason surveillance isn’t a catchall, Chertoff said, is because the phone records collection program combined with online data collection generates a massive amount of data. Even if authorities are tipped off before an attack, as was the case with Tamerlan Tsarnaev before the Boston Marathon bombing, someone still needs to identify that data and act before it’s too late.

“There’s not a technological solution,” Rich Roberts, spokesman for the International Union of Police Associations, which works with roughly 500 police departments, said. “Any time you have that kind of violence, or any kind of violence, including terrorism, it is often because there’s a certain mental instability in many of these people. And the fact is that if police have a reason to monitor communications they can already go through the courts to do it.” 

A total of 462 people have died and 1,314 been wounded in U.S. mass shootings (when four or more people are shot) since the beginning of 2015, according to records from gun-tracking databases reviewed by the New York Times. Easy access to lethal weapons, the absence of a national gun registry and media sensationalism have all been cited as contributing factors. The suspects in this case weren’t under FBI surveillance and purchased their weapons legally. 

Law enforcement officials previously told reporters Farook attended the office party before storming out in anger, then returned with his wife to carry out the attack, which was not “a spur-of-the-moment thing,” according to the San Bernardino police chief. Even if the suspects' communication was under surveillance, police would have needed to react almost instantaneously to stop the attack.

“The sad fact is that even if the NSA was scanning Americans’ communications for keywords or something, most teenagers probably use those words every day when they’re playing video games,” said Todd Morris of Brickhouse Security, which develops surveillance equipment with U.S. government contractors. “That wouldn’t work for us.”

As the U.S. intelligence community determines how long it will take to extract intelligence from telecommunication companies, the answer to stopping this kind of attack could lie with artificial intelligence. There’s no one way to unite all that disjointed data, but more companies are investing in technology that enables them to sweep social media and other open Internet channels for activity that could indicate an attack is imminent.

“Instead of finding computer viruses with artificial intelligence we’re finding human viruses,” said Akli Adjaoute, CEO of the artificial intelligence company Brighterion, which also counts three-time Interpol Secretary General Raymond Kendall on its executive team.

After years of scouring the Web for human traffickers, money laundering rings and illicit credit card transactions, the company is working with countries throughout the European Union to deploy machine learning on European social media users.

“We can process 8,000 transactions in less than 6 milliseconds,” Adjaoute said. “We can use all available information on Twitter, look to see if a bad guy’s face matches any facial recognition and eventually find out if the guy is corresponding to any bad people.”