Three of the 11 Secret Service employees being investigated for allegations of sexual misconduct in Colombia are leaving, the agency said Wednesday evening.

Although the Secret Service's investigation into allegations of misconduct by its employees in Cartagena, Colombia, is in its early stages, and is still ongoing, three of the individuals involved will separate or are in the process of separating from the agency, a statement from Assistant Director Paul Morrissey said, Reuters reported.

One supervisor was allowed to retire, another supervisor was proposed for removal for cause, and a third employee resigned, the Secret Service said.

“These guys have the clearest cases,” a person briefed on the matter told the New York Times.

The government is investigating an incident in Cartagena involving Secret Service employees and military personnel in which as many as 21 women were brought back to their hotel last week, before the arrival of President Barack Obama for a summit.

The remaining eight Secret Service employees are still on administrative leave with their security clearances suspended.

The agents were interviewed at Secret Service headquarters and offered lie detector tests Tuesday, CBS reported. 

These are the first steps, Rep. Peter King, R-N.Y., chairman of the House Homeland Security Committee, which oversees the Secret Service, told Fox News. King said the agency's director, Mark Sullivan, took employment action against the three people he believes the case was clearest against. But King warned: It's certainly not over.

The Secret Service continues to conduct a full, thorough and fair investigation, utilizing all investigative techniques available to our agency, the agency statement said. This includes polygraph examinations, interviews with the employees involved, and witness interviews, to include interviews being conducted by our Office of Professional Responsibility in Cartagena, Colombia.

The Secret Service has also expanded its internal investigation to look at what other personnel have done on trips, the person said. So far, investigators haven't uncovered anything similar to what apparently happened in Colombia last week.

King said he has assigned four congressional investigators to the probe. The House Committee on Oversight and Government Reform, led by Rep. Darrell Issa, R-Calif., sought details of the Secret Service investigation, including the disciplinary histories of the agents involved. But the committee was likely to have very few if any public hearings on the matter, Issa told reporters.

He noted that the Secret Service was already investigating and they are likely to be more harsh to their own people than we would be.

Much of the information that the House panel might unearth shouldn't be made public because of the nature of the Secret Service's work, he added.

Issa and the committee's ranking Democrat, Elijah Cummings, wrote to Sullivan asking for a detailed description of the misconduct, as well as an accounting of all U.S. government personnel who were involved in or knew about the alleged misconduct, and a timeline of the agency's response.

They also asked for summaries of any disciplinary actions taken against the agents involved since 2002, and what steps the agency intended to take to prevent a recurrence.

The incident in Cartagena raised questions about the agency's culture, lawmakers said in the letter.

It is troubling because Secret Service agents and officers made a range of bad decisions, from drinking too much, to engaging with prostitutes, to bringing foreign nationals into contact with sensitive security information, to exposing themselves to blackmail and other forms of potential compromise.

Sen. Susan Collins, R-Maine, the ranking Republican on the Senate Homeland Security Committee, said she told Sullivan she found it hard to believe the episode was the only one of its kind, because there were too many people involved.

He said they were scrubbing the files and looking at whether there were any hints that there had been previous incidents, Collins told reporters separately.

Collins, spoke to Sullivan about the episode on Monday and Tuesday, when she pressed him to look into the files.

Think of all the missions and countries that the Secret Service visits in advance of the president's trips, she said.

I think they should look at disciplinary records, at whether supervisors were -- had admonished (them) even informally, she said. My instinct is that this was not one-time.

Republican presidential candidate Mitt Romney told radio host Laura Ingraham Wednesday that I'd clean house at the Secret Service.

The right thing to do is to remove people who have violated the public trust and have put their play time and their personal interests ahead of the interests of the nation, Romney said.

While Romney suggested to Ingraham that a leadership problem led to the scandal, he told a Columbus, Ohio, radio station earlier that he has confidence in Sullivan, the head of the agency.

I believe the right corrective action will be taken there and obviously everyone is very, very disappointed, Romney said. I think it will be dealt with (in) as aggressive a way as is possible given the requirements of the law.