Confidence among U.S. homebuilders plummeted in July to its lowest level seen since the start of the pandemic, as potential buyers continue to retreat from the market due to higher prices.

On Monday, the National Association of Home Builders/Wells Fargo Housing Market Index, a survey designed to gauge market conditions, found builder sentiment dropped 12 points to 55.

Any rating above 50 is considered positive under the ranking's criteria, but the drop was the single largest recorded outside of April 2020 when the score nosedived 42 points over the pandemic.

Homebuilders are currently stuck in an unfavorable operating environment. Mortgage rates for homes have been on the upswing in response to the Federal Reserve's more hawkish approach to interest rates, which have risen three times since March. The sector has also continued to struggle with supply chain disruptions that have driven up the costs of construction materials and the final cost of homes for buyers.

“Affordability is the greatest challenge facing the housing market,” Robert Dietz, NAHB’s chief economist, told CNBC. “Significant segments of the home buying population are priced out of the market.”

Weakness in the housing market from these imbalances has contributed to rising concerns of a looming recession.

Fed Chairman Jerome Powell acknowledged in June that the housing market was softening in response to the higher mortgage rates that came with the Fed’s choices.

But Powell conceded that he did not yet know what the total impact rising interest rates would have on housing prices or residential investment in the market. He described it as still remaining "very tight."

“We need to get back to a place where supply and demand are back together, and where inflation is down low again and mortgage rates are low again," Powell said.