KUALA LUMPUR (Reuters) - Malaysia's highest court will deliver a decision on Tuesday on an appeal against a sodomy conviction by opposition leader Anwar Ibrahim, with a ruling against him likely meaning a five-year prison term that would stymie his political ambitions.

Anwar was the ruling party's rising star in the mid-1990s before he fell out with then Prime Minister Mahathir Mohamad.

Since then, the former deputy prime minister has been beset by legal problems and spent several years in prison after being convicted of corruption and an earlier sodomy charge.

But the charismatic Anwar, who heads a three-party opposition alliance, remains the greatest threat to Malaysia's political establishment.

His lawyers appealed against a conviction last March of sodomizing a male political aide, and a sentence of five years in prison. Sodomy is illegal in Muslim-majority Malaysia.

Anwar denied the charge and said it was politically motivated.

A rejection of his appeal could end his political career and may spark protests after opposition gains in a 2013 general election raised the possibility of a genuine challenge for the Barisan National coalition that has ruled since 1957.

A conviction would disqualify the 67-year-old from political office and contesting the next election that must be held by 2018. The prosecution had asked for a longer prison term.

Speaking to journalists last week, Anwar said he was cautiously optimistic.

"If the decision is based on law and facts, there is no possibility of a conviction," he said.

"Authoritarian figures think the best way to deal with dissidents is to jail them or shoot them. But throughout history this always backfired."

Human Rights Watch decried what it called Anwar's "selective persecution".

"PM Najib's government wants to push him off the political process," said Phil Robertson, the group's deputy director for Asia.

Prime Minister Najib Razak's government has rejected any suggestion of interference in the case saying Malaysia has an independent judiciary and it had no role in any court decision.

Police have called on Anwar's supporters to maintain calm and security at the court has been tightened. Last year, he appealed to supporters to shun violence.

Some ruling party members and analysts have played down fear of upheaval if the ruling goes against Anwar.

"Anwar's political image is not anywhere near close to what it was in the '90s," said Wan Saiful Wan Jan, of the Institute for Democracy and Economic Affairs.

"The country has moved on."

Anwar's jailing could herald the disintegration of the three-party alliance he leads. It is paralyzed with infighting and a meeting on Sunday to iron out differences and find a possible successor to Anwar was inconclusive.