Warren Buffett said he has no plans to soon exercise Berkshire Hathaway Inc's warrants to buy $5 billion of Goldman Sachs Group Inc stock, although he could make a big profit by doing so.

Berkshire got the warrants in September when it also bought $5 billion of Goldman preferred shares, which throw off a $500 million annual dividend.

The warrants let Omaha, Nebraska-based Berkshire buy Goldman common shares at $115 each at any time until October 1, 2013. With Goldman's stock having closed at $165.45 on Thursday, those warrants are worth well over $2 billion.

Buffett is sitting tight.

We will hold the warrants, Buffett said on Fox Business Network. Every instinct in my body tells me that we will want to hold those warrants until they're very close to their expiration date. The preferred pays us the dividend and the warrants are going to make us the money.

Goldman is the largest financial services company to exit the U.S. government's bank bailout program.

Earlier this week, it paid $1.1 billion to buy back warrants issued to the Treasury Department. The government said it got a 23 percent annualized return on its Goldman investment.

For a while, Goldman had looked like one of Buffett's lesser ideas, as its shares fell below $48 in November.

Yet its recovering stock price signals investors' belief that Goldman still deserves much of its luster as one of the world's most aggressive and profitable banks.

Buffett, the world's second-richest person, has had less success with a purchase of preferred stock and warrants in General Electric Co .

Berkshire in October bought $3 billion of GE preferred shares yielding 10 percent and warrants to buy $3 billion of GE stock at a $22.25 strike price. GE shares closed Thursday at $11.95, leaving the warrants out of the money for now.

Class A shares of Berkshire rose $1,050 to 94,550 in afternoon trading on the New York Stock Exchange.

(Reporting by Jonathan Stempel; additional reporting by Steve Eder)