It's time to ditch hedge funds. That’s the message emerging from an audit of Utah’s $32 billion public employee pension fund, which will join a growing list of retirement systems reconsidering their exposure to hedge funds and other niche investments.

Over the past decade, the Utah Retirement Systems piled increasingly into alternative assets, a category that includes hedge funds, private equity firms and real estate. Utah’s share of alternative investments tripled between 2004 and 2013, rising to 40 percent from 13 percent. The median pension fund had about 22 percent of its assets in alternatives in 2013.

If Utah had maintained its 2004 portfolio, however, its public employees would have another $1.35 billion in the kitty, independent consultant Chris Tobe found. As URS shifted into alternative investments, it moved out of a stock rally that far exceeded average hedge fund returns.

The end result, says a report authored by Utah's auditor general: Employees have to pay more out of pocket to recoup pension fund shortfalls.

Tobe, an independent consultant who has previously written about mismanagement in Kentucky’s underfunded public pension system, also highlighted the fees demanded by hedge funds. Investment fees withheld by managers totaled $194 million in 2013, largely due to outsized hedge fund allocations.

A 2014 analysis by CEM Benchmarking found that Utah’s investment fees came in 58 percent higher than similar state pension funds.

Utah’s public pension board originally justified its move into alternative investments as a way of diversifying its assets and mitigating the risk of a big stock market plunge. Utah hasn’t been alone: The share pension funds’ alternative investments nationwide grew steadily over the last 15 years.

When the financial crisis hit in 2008, hedge fund investments cushioned against big stock market losses. URS would have lost around $436 million more in 2008 had it maintained its stock-heavy balance of funds from 2004.

But those savings were more than offset by missing out on a historic stock rally and paying growing fees to hedge fund managers, Tobe reported to Utah’s legislative auditor general. “URS held up better than most in 2008, but lagged in the bull markets,” he wrote.

Tobe advised the state to slowly pare its alternative asset load to levels on par with other state systems, a recommendation passed on to the legislature in the auditor general’s report.

Whether Utah acts on Tobe’s recommendation, it joins a growing list of states taking a second look at their growing alternative investments. Earlier this year, Pennsylvania Gov. Tom Wolf singled out fees paid out to Wall Street as a top pension concern. In 2014, California’s massive CalPERS announced it would pull its $4.5 billion in hedge fund investments.

In a 10-year time frame, Utah has performed about average compared with other state retirement systems. But in the last five years, URS lagged behind members of peer groups like Arizona and Oregon, which grew 14.2 percent and 13.8 percent, respectively, compared with URS’ annualized 12.1 percent gains.