Rescue workers Wednesday halted their search for the missing passengers of the Costa Concordia cruise ship that capsized Friday off the west coast of Italy after the position of the ship moved, said the Italian coast guard.

With cold temperature and freezing waters, the rescuers are finding it difficult to search for more survivors Wednesday morning as about 24 people remain missing.

Tuesday, rescuers pulled five bodies from the wreckage, raising the death toll to 11.

The rescue workers are racing against time as the weather is set to turn stormy Thursday with waves of up to two meters forecast, Sky News reported.

Meanwhile, a 4-minute audio clip of a conversation between the Italian Coast Guard and the captain of the ship was released Tuesday which reveals that the captain refused the guards orders to return to the ship.

The prosecutor has accused Francesco Schettino, captain of the capsized ship, of manslaughter, causing a shipwreck and abandoning the ship Friday night before all passengers were evacuated.

But Schettino said that he was on the ship only, complaining it was too dark on the ship. He was questioned Tuesday for three hours before an Italian judge ordered him under house arrest, according to his attorney.