Carl Icahn, chairman of Icahn Enterprises, speaks July 15, 2015. Adam Jeffery/CNBC/NBCU Photo Bank via Getty Images

American International Group Inc. capitulated in its boardroom fight with activist investor Carl Icahn, nominating both his representative and billionaire John Paulson to its board, as it unveiled a bigger-than-expected quarterly loss.

Shares of the company, which boosted its share buyback program and dividend, rose marginally in after-hours trading.

The agreement with Icahn comes just two days ahead of AIG's deadline for shareholders to submit board candidates.

Icahn has been threatening a proxy fight at AIG, the biggest U.S. commercial insurer as he pushes to break it up into three smaller companies.

"We continue to believe that smaller and simpler is better," the investor said in a statement on Thursday.

Sandler O'Neill & Partners LP analyst Paul Newsome said the deal does not signal the end of the tussle between AIG's management and Icahn.

"If anything, it will continue behind the scenes rather than in public," he said.

AIG said it agreed to nominate Paulson, a billionaire investor and president of Paulson & Co, and Samuel Merksamer, a managing director at Icahn Capital LP, to its board.

Merksamer, 35, already sits on the boards of 10 companies, including Cheniere Energy Inc., Transocean Partners LLC and Hertz Global Holdings Inc., according to Reuters data.

The strategy proposed by Icahn is incredibly difficult to achieve, said Meyer Shields, an analyst at Keefe, Bruyette & Woods, North America. "There are things AIG can do to increase value - mostly doing a lot less of the things that they are frankly terrible at."

AIG reported an after-tax operating loss attributable to the company of $1.35 billion, or $1.10 per share, for the fourth quarter ended Dec. 31, hurt by weak underwriting and lower returns on investments in a turbulent market.

Analysts on average had estimated a loss of 93 cents per share, according to Thomson Reuters I/B/E/S.

The company is cutting costs as its underwriting business struggles with falling rates for commercial property and casualty insurance.

The insurer, which has frozen its employee pension plan, is looking to cut its gross general operating expenses by another $1.6 billion by the end of 2017.

AIG, which traces its roots to a two-room office in Shanghai in 1919, said it would buy back an additional $5 billion of its shares and raised its quarterly dividend to 32 cents per share from 28 cents. AIG repurchased about $10.7 billion of shares in 2015.

The company's shares closed at $50.52 on the New York Stock Exchange.

(Reporting by Sudarshan Varadhan and Sweta Singh in Bengaluru; Editing by Anil D'Silva and Saumyadeb Chakrabarty)