Victor Castro Jr. grew up in a modest, three-bedroom house in the heart of a middle-class neighborhood in suburban Phoenix.

But when his father lost his job three years ago in the recession and their property's value plunged, the Castro family had to walk away from the cherished home where they had lived for 12 years.

"You don't realize how much a house can mean to you," said Castro, a professional boxer, who turned 19 years old on Sunday. "There were so many memories."

The Castro family are among U.S. Hispanics and other minorities whose wealth slipped further behind that of white households during the recent housing crash and economic slump, according to a study released on Tuesday.

The Pew Research Center found whites' median wealth was a record 20 times that of black households and 18 times that of Hispanics in 2009 -- the largest gap in the 25 years the government has been keeping records.

The report, based on Census Bureau figures from 2009, said the inflation-adjusted median net worth of Hispanics plunged by 66 percent from 2005 to 2009, when the housing crisis and recession battered the country.

Black households' wealth, meanwhile, plunged by 53 percent during the same period.

That compared to a 16 percent drop among white households, researchers said.

"Everyone took a hit as measured by their net worth, it's just that Hispanics and blacks took a bigger hit than whites and fell farther behind ... than at any time in recorded history," said Rakesh Kochhar, one of the report's authors.

The typical black household had $5,677 in wealth in 2009, Hispanics had $6,325 and whites registered $113,149. Wealth was calculated as assets minus debts.

Kochhar said the housing bust was the biggest single reason for the heavy knock taken by minorities. Particularly vulnerable were Hispanics living in states with large minority populations including Arizona, California and Florida, which were also hit-hard by the foreclosure crisis.

"They are much, much more likely to be living in areas that went on that roller coaster ride with price increases and big price decreases," he said.

Kochhar also said the equity in a home generally accounts for a larger share of the net worth of Hispanic and black households.

"They don't have too much in terms of financial assets, so they were more dependent on the housing market," he said.

Although Castro's family have been floored by the recession and now live in a rented apartment, the junior welterweight said they remained determined to pick themselves up again.

"We all have our ups and downs, but it's how you fight back that matters," said Castro, who turned professional two months ago. "We won't be down for long."