The Secret Service leaked unflattering confidential information about one of its most dogged congressional critics to the press, according to an inspector general's report released Wednesday. Rep. Jason Chaffetz, R-Utah said the move was an attempt to “intimidate and embarrass” him.

The Secret Service leaked the now-congressman's personnel file, which showed that his application to join the organization in 2003 had been rejected.

The Secret Service apologized Wednesday to Chaffetz, who, as chairman of the House Committee on Oversight and Government Reform, has been probing allegations of misconduct that have dogged the agency in recent years.

“Some information that he might find embarrassing needs to get out,” Secret Service Assistant Director Edward Lowery wrote in an e-mail to a colleague on March 31, commenting on information from Chaffetz's personnel file which, according to the Washington Post, was being widely circulated within the organization at the time. “Just to be fair,” Lowery added.

Two days after Lowery's email, details of Chaffetz's unsuccessful application to the Secret Service were published in the media.

"It's a bit scary. If they would do this to me, I just, I shuddered to think what they might be doing to other people," Chaffetz told NBC News. "I'd like to tell you how tough I am, but it's scary, and it's intimidating, and I will continue to investigate the Secret Service and others, but this should have never ever happened."

The report by John Roth, inspector general for the Department of Homeland Security, found that between March 24 and April 2, 2015, 45 Secret Service employees accessed “sensitive personal information” on Chaffetz nearly 60 times. Only 4 of the 45 employees had “an arguable legitimate need to access the information,” the report found.

Commenting on the report, Rep. Elijah Cummings of Maryland, the Oversight Committee's top Democrat, said in a statement: "I believe in fundamental fairness, and those who are unwilling or unable to meet the highest of ethical standards should not be a part of the Secret Service."

Chaffetz's committee initiated an investigation into the Secret Service earlier this year, following a series of high-profile blunders on the part of the organization, including an incident in which a knife-wielding man entered the White House, and another in which an armed man was allowed to enter an elevator with President Barack Obama, Politico reported.

The agency has also faced allegations that a vehicle driven by two of its members, who reports say had been drinking, crashed into a security barrier outside the White House earlier this year. In addition, in 2013, several agents were found to have hired prostitutes during a visit by President Obama to an international summit being held in Colombia.