Over the years, the number of detainees held at the U.S. detention center at Guantanamo Bay, Cuba, has fallen from hundreds of individuals to just 91. If President Barack Obama is able to push forward a long-anticipated plan to shut down the facility, that population could soon fall to zero, as he announced Tuesday his intentions to see remaining detainees transferred to prisons on U.S. soil.

In the past, Guantanamo Bay has made headlines for reports of harsh interrogation tactics — including waterboarding — at the prison facility; at least detainees died there, seven of them suicides. Obama vowed to shut the center down during his 2008 campaign and ordered it just after taking office, but little progress was made. He found it difficult to garner the necessary political support as his proposals came under harsh scrutiny in Congress from those who disagreed with prosecuting detainees in the U.S.

So who is actually being held there?

Of the detainees, 35 have already been ordered to be released to third-party countries and just a handful have had charges brought against them. Many of the others remain in a legal limbo: held without charge but periodically reviewed to determine whether they are eligible to stand trial. Many have gone on hunger strike in the past, protesting as many as 15 years in detention without charges.

Officials have been concerned about bringing to trial individuals who they say may pose a threat to the U.S. but against whom there's insufficient evidence to make that case in court. Human rights advocates, however, criticize continued detentions for denying individuals the right to a fair trial.

Among those being held are high-profile detainees like Khalid Shaikh Mohammad, the Pakistani citizen said to be “the principal architect” of the Sept. 11, 2001, attacks in New York City, according to the 9/11 Commission Report. He has been subjected to waterboarding during interrogations, and has been held since 2006.

Another high-profile detainee, Abu Zubaydah, is suspected of being an al Qaeda operative who also had a hand in the 9/11 attacks.

Ahmed Umar Abdullah al Hikimi, a Yemeni, is suspected of once being Osama Bin Laden’s bodyguard and fighting U.S. forces in Afghanistan. Believed to be a source of information about other targets, he was among a group of prisoners subjected to the harshest interrogation measures.

But a vast majority of those being held at Guantanamo Bay are not high-profile suspects and many were never known to be allied with al Qaeda. The detainees reportedly come from more than a dozen countries and more than half are from Yemen. Thirty-five of them have already been cleared for release. 

"This is about closing a chapter in our history," Obama said during a speech at the White House Tuesday.