The political fallout from CIA Director David Petraeus’ resignation only looks to be spreading with the news that an FBI whistleblower told House Majority Leader Eric Cantor of the general’s affair a week before the president had any inkling.

Cantor, R-Va., was apparently concerned about the potential for a security leak and passed the information to his chief of staff Steve Sombres, who then filled in FBI Director Robert Mueller on the accusation, according to The New York Times.

“I was contacted by an FBI employee concerned that sensitive, classified information may have been compromised and made certain Director Mueller was aware of these serious allegations and the potential risk to our national security,” Cantor said in a statement.

Politico notes that the revelation that more people knew about the scandal so early raises questions about not only the ripple effect, but how tight the FBI keeps its ship. Petraeus’ once-revered reputation is at stake.

“People think that because it’s the CIA director, it must involve bigger issues,” one government source told the Times. “Think of a small circle of people who know each other.”

Members of the CIA’s brass are scheduled to meet with House Intelligence Committee leaders next week and update them on the investigation.

Cantor and presumably his top aides knew about the allegations days before director of National Intelligence James Clapper was informed on Tuesday. Petraeus informed President Barack Obama during a brief meeting Thursday, and the president accepted his resignation Friday afternoon.

When Sombres told the FBI about the information from the whistleblower he was told an investigation into the matter was already in progress. Major media outlets have reported Petraeus had been having an affair with his biographer Paula Broadwell, a married mother of two. Scott Broadwell, Paula’s husband, is an interventional radiologist.

Government security reportedly found out about the matter when they were investigating Broadwell for sending “harassing” messages to other women in Petraeus’ life. The person receiving the messages has not yet been identified, according to The Boston Globe, but some outlets have expressed doubt they were sent to Petraeus’ wife Holly, whom he’s been married to for 37 years.

“It didn’t start with Petraeus, but in the course of the investigation they stumbled across him,” a congressional official said, adding the inquest began with “two women.” “We were stunned.”

The FBI was less interested in finding out about Petraeus’ affair, which is not a criminal matter, but whether his Gmail account had been compromised. The New York Daily News unsuccessfully tried to find Broadwell and her husband at their North Carolina home, but instead were met with neighbors also curious about Petraeus’ emails.

“The question here is not just the affair, but what she knows,” one said. “How much access she had to him.”