Foreign banks are downsizing their operations in Japan, with Bank of America Corp. (NYSE:BAC) being the latest example, and raising fears that Tokyo’s status as a global financial center is under threat.

Before the financial crisis, foreign banks looked to Japan’s 1,500 trillion yen ($17.94 trillion) in household financial assets and had hoped to claim a piece of this largely untapped market for private banking services. But this year’s recession and weak market activity have made it difficult to generate enough revenues to cover the high costs of operating there.

The Japanese yen has risen 24 percent against the U.S. dollar since the September 2008 collapse of Lehman Brothers. Meanwhile, data from the Bank of Japan shows that 56 percent of Japanese household assets are still held as cash or bank deposits.

Tokyo’s city ranking among 77 global financial centers has fallen to No. 7, from No. 5 in 2011, according to the latest Global Financial Centers Index released in September. Moreover, Tokyo was not among the 10 centers voted most likely to see new offices open over the next few years.

As part of its global strategy to move away from international wealth management business and focus resources on markets where it has more competitive strengths, New York-based Bank of America is disposing of its Japanese private banking joint venture with Mitsubishi UFJ Financial Group Inc. (TYO:8306) to the Japanese group for about 39 billion yen.

The joint venture -- Mitsubishi UFJ Merrill Lynch PB Securities Co. -- was formed in May 2006 by MUFJ and Merrill Lynch Co. The joint venture was integrated into Bank of American’s operations after it acquired Merrill Lynch during the height of the financial crisis in 2008.

The decision to sell the joint venture in Japan follows Bank of America’s sale of its loss-making, non-U.S. wealth management business to Swiss private bank Julius Baer Group Ltd. in August.

Several other major investment banks have also announced plans to either reduce headcount in their Japanese operations, or close branches in the world’s third-largest economy.

Regulatory filings show that Goldman Sachs Group, Inc. (NYSE:GS) cut staff numbers in Japan by 14 percent to 847 in the 12 months to March 2012, compared with the previous financial year. Meanwhile, UBS AG (NYSE:UBS) trimmed the number of employees based in Japan by 12 percent to 768 over the same period.

HSBC Holdings PLC (LON:HSBA) and Standard Chartered PLC (LON:STAN) have both said they will pull out of the market serving Japan’s high net-worth individuals, according to the Financial Times. HSBC has abandoned both premier and private banking in the country, and Standard Chartered is exiting the priority banking business in Japan next spring.

In June, Lloyds Banking Group PLC (LON:LLOY), which is 41 percent owned by the U.K. government, said it would sell its remittance business in Japan to Shinsei Bank Ltd. (TYO:8303) and wind down its fixed-term deposit book.

The Japanese administration under the Democratic Party hasn’t taken bold enough steps to bolster the financial sector, critics say, and the financial industry is banking its hopes on a return to power of the Liberal Democrats under leader Shinzo Abe in elections on Sunday.

Japan has entered its fifth recession in 15 years just days before the Dec. 16 election. Gross domestic product for the second quarter was revised down Monday to show a contraction of 0.1 percent, a trend that accelerated from July to September, leading to a contraction of 3.5 percent for the third quarter. Economists are becoming increasingly pessimistic and many expect GDP to contract again in the fourth quarter.

Shares of Bank of America Corp (NYSE: BAC), which have gained 90 percent this year, rose 0.2 percent at $10.56 in Friday afternoon session.