The vote came amidst widespread public anger over tough austerity measures and allegations of state corruption, with 235 lawmakers voting against the government -- four more than was needed to topple Ungureanu.
Opposition leader Victor Ponta said the vote represented the end of an abusive system that uses any weapon possible, according to the Associated Press.
Sometimes there is justice. Today there was justice, he added.
President Traian Basescu will now nominate a new prime minister.
The defeat is the latest in a series of setbacks for austerity-minded governments as they cut budgets and scale back spending in a bid to reduce national debts.
Romania's former PM Emil Boc called the result, a victory for opportunism in politics and party-swapping.
Boc, whose party was toppled in February after widespread street protests against austerity measures, added, I take responsibility for this failure.
Romania's currency, the Leu, fell on the news, hitting an all-time low of 4.401 against the euro.
In response, the International Monetary Fund said it would postpone a review of the country pending more information on the shape of the new government.
Elsewhere, the Czech government also faced its own no-confidence vote on Friday, whose budget-cutting cabinet is expected to survive.
The end result seems to ... echo what we have been seeing in other countries in terms of a popular move away from the parties that are pushing for austerity, Koon Chow, a strategist at Barclay's Capital, told Reuters.