Brent and U.S. crude oil futures rallied more than $2 on Wednesday, with Brent over $123 a barrel, helped by a weekly draw in U.S. crude inventories that confounded expectations for a stock build, and by a weaker dollar.

ICE Brent crude was up $2.11 at $123.44 a barrel by 1506 GMT, after pushing to intra-day highs of $123.50.

U.S. crude was up $2.39 at $110.67 a barrel, after pushing to $110.89 in feverish trading after the bullish data from the Energy Information Administration (EIA).

The EIA showed a fall in U.S. crude stocks of 2.32 million barrels week on the week, compared with forecasts for a 1.1 million barrel build . Gasoline and distillates inventories were also down.

Both crude oil futures contracts extended gains after the data. Crude stocks fell for the first time in while, which took the market for a bullish surprise, said Mike Zarembski, an analyst at OptionsXpress in Chicago.

But Carl Larry, president of Oil Outlooks, pointed to a big drop in crude imports and suggested crude is now getting too expensive for refiners to buy. They are feeling the same sticker shock that consumers are feeling at the pump, he said.

A weaker dollar is also supporting the oil price.

Oil is up there with gold as an inflation hedge for investors, said Zarembski. Everyone is still afraid to be short with the situation in the Middle East.

The dollar index <.DXY>, which measures the greenback against a basket of currencies, was down 0.86 percent by 1458 GMT. A weaker U.S. currency can support dollar-denominated commodities by rendering them less expensive for other currency holders.

The dollar index has broken significantly below a long-term multi-year trendline, so we could see the selling accelerate, said GFT market strategist David Morrison. It's a shorthand for commodity traders to look at the dollar index, which gives some support to commodities going forward.

Spot gold prices breached $1,500 for the first time ever, and silver hit a 31-year high on Wednesday.

DEMAND DESTRUCTION FEARS

The International Energy Agency's executive director, Nobuo Tanaka, issued the latest in a series of warnings on the effect of strong oil prices on demand in top consumers the United States and China.

Producer group OPEC needs to boost output in June or July to douse further price rises, Tanaka said, adding that if crude prices stayed at $100 a barrel or more for the rest of 2011, the market could see demand destruction similar to that of 2008.

OPEC has to date declined to make a coordinated increase in supply, despite growing concerns about demand destruction as world oil prices rocketed up to 2-1/2-year highs of $127 a barrel earlier this month amid unrest in the Middle East and North Africa.

We don't have much fundamental news, but Saudi Arabia recently cut output in the last few weeks, which was not expected, and all in all prices could go up a bit, LBBW analyst Frank Schallenberger said.

(Additional reporting by Francis Kan in Singapore and Zaida Espana in London, editing by Anthony Barker)