By Jim Wolf and Jason Szep

FREDERICKSBURG, Virginia/ST.JOHN'S, Antigua, Feb 20 - Regulators seized Texas billionaire Allen Stanford's banks and companies in Antigua and Barbuda, the Caribbean state at the center of fraud charges against him, as the financier surrendered his passport to U.S. authorities.

Antigua's government, which in 2006 gave Stanford a knighthood, and the Eastern Caribbean Central Bank announced the takeovers on Friday, after a rush by depositors this week to withdraw their funds from Stanford banking affiliates.

Stanford, 58, was charged by the U.S. Securities and Exchange Commission (SEC) on Tuesday with fraudulently selling $8 billion in certificates of deposit with improbably high interest rates from his Stanford International Bank Ltd (SIB), headquartered in Antigua.

SEC spokesman Kevin Callahan said that Stanford and his two co-defendants, James Davis, SIB's chief financial officer, and Laura Pendergest-Holt, chief investment officer of a Stanford affiliate, had surrendered their passports in keeping with a judge's order.

On Thursday the FBI tracked Stanford down in Fredericksburg, about 50 miles south of Washington, to serve him with court papers, but a day later Stanford was nowhere to be seen in the historic Virginia town.

The FBI, Internal Revenue Service, SEC and banking regulators have been investigating allegations of fraud and other possible illegal activity by Stanford's companies for at least two years, the Washington Post reported in Saturday editions, citing people familiar with the matter.

The FBI was exploring bringing other charges against him, and the SEC was continuing its investigation, it said.

Meanwhile, the court-appointed receiver overseeing the assets of Stanford's customers -- Dallas lawyer Ralph Janvey -- said in a statement the customer accounts were frozen until legal claims could be sorted out.

Also, a federal judge presiding over the Stanford case, U.S. District Judge Reed O'Connor, was recused because a relative has substantial Stanford holdings, according to a court filing.

The Stanford scandal, hard on the heels of allegations that Wall Street veteran Bernard Madoff carried out a $50 billion fraud, has spooked international investors and triggered investigations in Latin America, Europe and the United States.

Peru said on Friday that it had started an investigation to see if the local unit of Stanford Group had engaged in money laundering.

Acting at the SEC's request, FBI agents served Stanford court orders and other documents on Thursday in Fredericksburg, Virginia. Stanford is not under arrest and his precise whereabouts remained unclear on Friday.

In Washington, Congressman Dennis Kucinich asked SEC Chairman Mary Schapiro to say by March 9 whether the agency suspended a 2006 investigation of Stanford. He also wants to know if it did so at the behest of another federal agency.

LIQUIDITY WORRIES

The Eastern Caribbean Central Bank announced the takeover of Stanford-controlled Bank of Antigua after what it called an unusual and substantial withdrawal of funds ... which has the potential to create severe liquidity problems for the Bank.

The central bank said publicity surrounding the Stanford case was threatening the interests of depositors and creditors.

Earlier on Friday, Antigua and Barbuda's bank regulator appointed a receiver to take control of Stanford International Bank (SIB) and Stanford Trust Company.

Stanford was the biggest private investor and employer in the tiny Caribbean state, for him just a part of a business empire that stretched from the United States to Latin America and Europe.

In Fredericksburg, news crews maintained a vigil outside the family home of a woman, Andrea Stoelker, a former town resident identified by British news reports as Stanford's girlfriend and president of the board of director of a cricket tournament that Stanford sponsored in Antigua.

In the U.S. Virgin Islands, where Stanford had plans to build a global management complex, authorities suspended his company from a tax-incentive program, a senior government official said on Friday.

We are cooperating with the SEC, Percival Clouden, executive director of the Virgin Islands Economic Development Authority, told Reuters by telephone from St. Thomas.

IMPACT ON SPORTS WORLD

The scandal has hit the world of international sport, where Sir Allen was a generous patron. Stanford once flew by helicopter to Britain's Lord's cricket ground with $20 million cash in sponsorship money.

The England and Wales Cricket Board (ECB) said on Friday it had terminated its contracts with Stanford's organization.

ECB was shocked by the charges filed against the Stanford organization and personnel earlier this week, ECB chief executive David Collier said.

The scandal was expected to hit his support of other sports like polo.

The Wall Street Journal reported on Thursday that U.S. federal prosecutors were investigating whether Stanford was operating a Ponzi scheme. In such a scheme, money from new investors is used to pay earlier investors.

In Venezuela, where banking regulators estimate citizens may have had more than $2 billion invested in Stanford offshore companies, investors were counting the cost of the scandal.

Middle-class and well-heeled Venezuelans said they invested with Stanford because they feared their money was not safe at home under socialist President Hugo Chavez.

We love that Stanford man like we love Chavez, one well-dressed young investor said bitterly, smoking at a Stanford office, where hundreds have lined up in recent days trying to get their money.

Venezuela's government seized Stanford's local commercial bank, a small operation holding local currency accounts, on Thursday after a run by customers using Internet facilities. The government promised to auction off the bank.

PREVIOUS DRUGS MONEY PROBE

The SEC filed charges in Dallas, Texas, on Tuesday against Stanford, two colleagues and Stanford International Bank Ltd, Stanford Group Co and Stanford Capital Management LLC.

The court-appointed receiver has moved to add Stanford Financial Group and Stanford Financial Group Bldg Inc to the complaint.

Across the Andean region and in Mexico and Panama, regulators have launched investigations into local Stanford affiliates, in some cases suspending their operations.

Federal agents raided Stanford Group Co offices in Miami, Houston and other U.S. cities earlier in the week.

Britain's Serious Fraud Office is monitoring a possible U.K. link after media reports that Stanford's books were audited in Britain.

In an interview on Thursday, Stanford's father, James, told Reuters that in the late 1990s, a Mexican customer of Stanford put $3 million into a Stanford bank for investment purposes. He said federal agents then approached Allen Stanford and said the money was from a Mexican drug cartel and had been laundered several times. James Stanford said his son had cooperated with the federal probe.

Antigua has faced U.S. scrutiny in the past for alleged money laundering activities and operations by suspected Russian shell banks.

Stanford's personal fortune was estimated at $2.2 billion last year by Forbes Magazine. He has donated millions of dollars to U.S. politicians.

(Additional reporting by Tom Brown and Jim Loney in Miami, Teresa Cespedes in Lima, Frank Jack Daniel and Ana Isabel Martinez in Caracas, Miles Evans in London, Ben Klayman in Chicago, writing by Pascal Fletcher in Miami; Editing by Toni Reinhold, Gary Hill)