U.S. property and casualty insurer Alleghany Corp made a surprise entry into the battle for Transatlantic Holdings with a $3.4 billion cash-and-stock deal to buy the reinsurer, squaring off against a hostile bid from Validus Holdings Ltd .

Alleghany's offer values Transatlantic at $59.79 per share, topping Validus' $55.59 a share offer, based on their closing prices Friday on the New York Stock Exchange.

But in Monday trading, Alleghany's shares were off as much 10 percent, while Validus' stock rose 4 percent. At those prices, Alleghany's offer was cut to $55.27, below the value of Validus' stock and cash bid that was now worth $57.37.

Transatlantic's shares rose 3 percent to hit a day high of $56, and were trading at $55.21 late morning.

Despite the volatile valuations, Alleghany's deal has the backing of Transatlantic's largest shareholder, Davis Selected Advisors and that could force Validus to raise its offer.

I think Validus will increase their offer, they have historically been aggressive in terms of pursuing acquisitions, Stifel Nicolaus analyst Meyer Shields told Reuters.

Validus, which is trying to unseat Transatlantic's board, declined to comment when contacted for this story.

Davis has been a vocal opponent of deals with Transatlantic's other suitors, which included reinsurer Allied World and National Indemnity, a unit of Warren Buffet's Berkshire Hathaway .

If Validus manages to win over the investment management firm or a majority of the other shareholders, it will be the company's second hostile takeover in two years, following its acquisition of IPC in 2009.

Under the terms of the Alleghany deal, Transatlantic shareholders will receive $14.22 in cash and 0.145 Alleghany shares for each share.

Alleghany said Joseph Brandon, former chief executive of Berkshire Hathaway's General Re Corp, would become chairman of Transatlantic and president of Alleghany Insurance Holdings

LLC.

Reuters had earlier reported that Brandon was working with Morgan Stanley on a possible bid for Transatlantic.

Besides the Brandon group, sources had also confirmed that Transatlantic had been in talks with Enstar, the J.C. Flowers- and Goldman Sachs-backed reinsurer that operates companies in run-off -- those not writing new insurance but only maintaining existing books of business until they expire.

Transatlantic is one of the cheapest companies in the industry and trades at a 20 percent discount to its book value compared with its peers, according to Thomson Reuters Starmine.

The company's long-tail insurance lines -- such as medical malpractice and workers' compensation -- are attractive to rivals more exposed to short-tail risks such as hurricane damage.

Alleghany said the acquisition would immediately add to its book value and earnings per share and bolster its capital base.

(Reporting by Tanya Agrawal and Jochelle Mendonca in Bangalore; Editing by Supriya Kurane, Anthony Kurian)