Yemeni forces loyal to outgoing President Ali Abdullah Saleh and opposition gunmen are withdrawing from the streets of Taiz city, an official said on Friday, as part of an effort to stop fighting that has threatened a political solution to the country's crisis.

Dozens have been killed in Taiz, Yemen's commercial capital, since Saleh signed a deal last month to give up power and end months of anti-government protests that have pushed the impoverished country to the brink of civil war.

The official said a committee set up to restore normality to Taiz was clearing away makeshift road blocks set up by Saleh opponents and loyalists during street battles, and overseeing their withdrawal from occupied buildings.

Separately, Saleh's General People's Congress (GPC) party said it would stop holding pro-government rallies after Friday prayers to show its commitment to a political solution to Yemen's woes.

The decision by the party's leadership is a new sign of the GPC's eager desire to act in the higher interest of the nation and to begin ending the political crisis, said a statement from the party, urging opposition parties to do the same.

Under the power transfer plan negotiated by Yemen's wealthy Gulf neighbours, the GPC and the opposition Joint Meeting Parties (JMP) agreed to divide up cabinet posts between them and form a national unity government to steer the country ahead of presidential elections next February.

The cabinet, which is due to be sworn in on Saturday, faces a host of challenges including a southern separatist movement, a rebellion in the north and a regional wing of al Qaeda that has exploited the upheaval to strengthen its foothold in Yemen.

Neighbouring Saudi Arabia and the United States, both targets of foiled attacks by al Qaeda's Yemen-based wing, fear the global militant network could use a gaping security vacuum to plot and perhaps carry out attacks on the region and beyond.

The interior ministry said it was making arrangements for the United Nations envoy who helped broker the power transfer deal to visit the north and south of the country, where central government control has been severely weakened.

Meanwhile, opposition leader Mohammed Basindwa, who is now prime minister, said his first foreign visit would be to oil-rich Saudi Arabia and the United Arab Emirates to ask for urgent support for Yemen's fuel and electricity needs.

(Reporting by Mohammed Ghobari; writing by Isabel Coles; editing by Philippa Fletcher)