U.S. President Barack Obama said he would be angry if allegations are proven to be true that Secret Service and military officials cavorted with prostitutes in Colombia ahead of his official visit there this weekend.

If it turns out some of the allegations that have been made in the press are confirmed, then of course I'll be angry, Obama said during a press conference with Colombian President Juan Manuel Santos Sunday, according to ABC News.

We're representing the people of the United States, and when we travel to another country, I expect us to observe the highest standards because we're not just representing ourselves, Obama said.

The president also called for a rigorous and thorough investigation in his first comments on the matter, and he noted he would reserve judgment until the probe was over.

Obama was in Colombia this weekend for the Summit of the Americas to discuss trade, labor, and other issues, but the visit has been overshadowed by an investigation that sent 11 Secret Service members home on administrative leave for the alleged misconduct. Five military service members are also under investigation, according to the Associated Press.

The men are accused of bringing prostitutes back to the five-star Hotel Caribe in Cartagena, which is within the security zone created for Obama's visit. U.S. Rep. Peter King, R-N.Y., told CNN that the Secret Service began investigating the incident because one of the women called local police to complain she had not been paid.

One of the women did not leave the room in the morning, King, chairman of the House Homeland Security Committee, told CNN. Visitors of guests were required to leave the hotel by 7 a.m.

It appears some of security personnel now under investigation allegedly went to a bar and brought back various friendly women who requested money when they got to the hotel rooms, a source familiar with the situation told Reuters.

Prostitution is legal in certain districts in Colombia. The officials are not being investigated on potential criminal charges, but on the bases of misconduct and violations of ethical standards.

They cannot put themselves in compromising positions where they're open to be blackmailed or threatened, King said.

U.S. Rep. Darrell Issa, R-Calif., told CBS' Face the Nation on Sunday that the prostitution allegations, if proven true, probably do not constitute an isolated incident.

The investigation will not be about the 11 or 20 or more involved -- it will be about how did this happen and how often has this happened before, Issa, a frequent critic of the Obama administration, told CBS. Things like this don't happen once if they didn't happen before.

A Secret Service representative told Reuters there was no evidence or information on any other incidents.

Ronald Kessler, author of In the President's Secret Service, called it clearly the biggest scandal in Secret Service history.

They don't have enough agents, they don't even put people through metal detectors sometimes because there's pressure to let everybody in, Kessler said on CBS This Morning Saturday. It's like letting passengers in an airplane without putting them through metal detectors.

Kessler added: They don't keep up-to-date with the latest firearms. They don't even do physical tests. So, it's a culture that leads to this kind of problem.