Libya's prime minister between 2003 and 2006 and its oil minister from 2006 to 2011, Ghanem worked as a consultant for a Vienna-based company, according to BBC News.
A police representative told the news agency there were no signs of violence on Ghanem's body, which was found in the river that flows through Vienna.
Ghanem, 69, defected from Libya during the civil war that toppled Gadhafi last year. At the time of his defection, he criticized the bloodshed in the country, saying the situation had become unbearable, BBC News reported.
A postmortem examination has been ordered.