Stocks fell on Tuesday, with the S&P 500 retreating from four-year highs after the U.S. Federal Reserve said it was less inclined to provide more economic stimulus.

Supportive central bank policies have been a primary catalyst for the S&P 500's surge of 30 percent since October, even though improving economic conditions have also played a part in the rally. Investors still expect a pullback, but markets have remained resilient, often cutting losses going into the close, as they did both Monday and Tuesday.

Sectors tied to growth were the big losers of the day, with energy shares <.GSPE> down 1 percent, and materials <.GSPM> off 0.9 percent. These sectors and the broader market extended losses after the release of the Fed's minutes, though they subsequently rebounded. The day's strongest performer, utilities <.GSPU>, is considered a defensive play.

The minutes from the Fed's March meeting noted a couple of committee members thought more stimulus might be needed if the economy loses momentum. The January meeting's minutes said a few members saw the need for more easing.

There have been signs of strength in manufacturing and employment. That has the Fed taking a wait-and-see approach on the sidelines, which is certainly a disappointment for a lot of investors, said Eric Teal, chief investment officer of First Citizens Bancshares in Raleigh, North Carolina, who helps oversee $21 billion in assets.

In the last week, economic figures have been less bullish for markets, making the less supportive words from the Fed a disappointment. However, U.S. auto sales posted their best quarter since 2008, another sign of recovery in demand.

However, General Motors Co shares slumped 4.6 percent to $25.54 as the increase in its sales trailed other automakers.

In contrast, Ford Motor Co rose 0.2 percent to $12.64.

Financials <.GSPF> fell 0.7 percent, with Morgan Stanley off 2.2 percent at $19.37 after the Federal Reserve said it was taking an enforcement action against the company for the way one of its mortgage servicing units handled home loans.

The Dow Jones industrial average <.DJI> declined 64.94 points, or 0.49 percent, to 13,199.55 at the close. The Standard & Poor's 500 Index <.SPX> slipped 5.73 points, or 0.40 percent, to 1,413.31. The Nasdaq Composite Index <.IXIC> dropped 6.13 points, or 0.20 percent, to 3,113.57.

Despite the day's declines, the Dow was still up 8 percent for the year while the S&P 500 remained up 12.4 percent and the Nasdaq was up 19.5 percent for 2012 so far.

Earlier, data from the Commerce Department reinforced the view the domestic economy was slowly improving as new orders for U.S. factory goods rebounded in February, although the increase was short of expectations.

U.S. auto sales jumped more than 15 percent in March, wrapping up the best quarter for U.S. vehicle sales since 2008.

About 62 percent of stocks listed on the New York Stock Exchange closed in negative territory while on the Nasdaq, two-thirds of shares ended lower.

Volume was light, with about 6.75 billion shares traded on the New York Stock Exchange, the American Stock Exchange and Nasdaq, below last year's daily average of 7.84 billion.

(Editing by Jan Paschal)