Bernard Madoff lost his bid on Friday to be released from jail until his June sentencing for Wall Street's largest-ever investment fraud.

The confessed swindler had challenged a judge's decision to immediately jail him after his dramatic guilty plea last week, but a federal appeals court denied his request for release after finding he was a potential flight risk and could have a secret pile of money to help him escape.

The ruling means Madoff might never again see the luxury Manhattan penthouse where he had been placed under home confinement for three months following his December 11 arrest.

He is now being held in a small concrete cell in a downtown Manhattan jail after confessing to a fraud that prosecutors say bilked investors of as much as $65 billion over 20 years. When the one-time Nasdaq chairman is sentenced, scheduled for June 16, he could be ordered to prison for the rest of his life.

A three-judge panel of the appeals court said Madoff, 70, could try to flee if allowed out on bail, given his age and potential for a long prison sentence. He faces up to 150 years in prison after pleading guilty to fraud, money laundering and other charges.

Madoff could have stashed money abroad to finance a potential escape, the U.S. Court of Appeals for the Second Circuit panel wrote in its ruling upholding District Judge Denny Chin's decision last week to jail Madoff immediately.

The defendant has a residence abroad, and has had ample opportunity over a long period of time to secret substantial resources outside the country, the judges said.

Madoff owns a home in France. While his attorneys argued his assets have been frozen and he has made no effort to flee, the appeals court said Judge Chin was not required to treat this defendant's financial representations as reliable.

Madoff's attorneys had sought to allow him to remain out of jail on bail until sentencing, as is often permitted for white-collar criminal defendants who are not considered violent or risks of flight.

We are disappointed and we respectfully disagree with the court's decision, but the court has ruled, defense attorney Ira Lee Sorkin said.

Madoff could potentially seek a rehearing before the full appeals court, but such a bid is considered a long-shot and there would be no guarantee the full court would agree to even hear the case.

Sorkin declined comment on possible next steps and would not discuss how Madoff is faring in jail.

The ruling came a day after the appeals court held oral arguments on Madoff's bail appeal.

Prosecutor Marc Litt had argued that while Madoff was granted bail immediately following his arrest -- a decision that enraged many of his defrauded clients, who wanted him incarcerated immediately -- his guilty plea had changed the landscape and raised the incentive for the swindler to flee instead of go to prison.

Litt said that while Madoff had been under constant monitoring at his apartment under his bail terms, there were at least three exits in Madoff's building that could be used to escape, and security guards can fall asleep while watching video monitors and potentially be bribed.

(Reporting by Martha Graybow, editing by Dave Zimmerman and Gerald E. McCormick)