Larry Robbins, founder, portfolio manager and CEO of hedge fund Glenview Capital Management, speaks at the Sohn Investment Conference in New York. Reuters

Hedge fund managers are adjusting to a new reality: losing.

Battered by volatile markets, bashed by populist politicians and squeezed by an increasingly crowded market, managers in the hedge fund industry are fighting to retain their elite image, as well as their trillions of dollars in investment assets.

A challenging trading climate in 2015 left a composite index of hedge funds down 0.9 percent for 2015. By comparison, the Standard & Poor’s 500 inched up 1.38 percent for the year. The market pain continued into the first quarter of 2016, when investors pulled $15.1 billion out of the hedge fund industry, the largest outflow since 2009.

Pain was still in the air Wednesday at the Sohn Investment Conference, an annual gala in New York City devoted to the business of managing other people’s money. Though the event exists ostensibly to raise funds for cancer research, its main attraction is the slate of hedge fund luminaries who descend every year to offer up their top investment ideas.

Looming over the event this year was the cloud of doubt that has darkened the hedge fund world over the past year.

“Hedge funds have been playing dodgeball,” Larry Robbins, founder and CEO of Glenview Capital Management, told his fellow speculators Wednesday. “Even hedge funds have been criticizing hedge funds.”

Those critics include Steven Cohen, chief executive of Point72 Asset Management, who blamed overpopulation for industry ills at a conference in Los Angeles on Monday. “It’s not easy to find great people,” Cohen said. Barred from managing outside money after his former firm pleaded guilty to insider trading violations, Cohen lamented the lack of qualified personnel available to his new efforts. “Talent is really thin.”

The number of hedge funds has multiplied more than 16 times over in the past 25 years, according to the data group HFR. Growth in the industry has plateaued in recent years at around 10,000 firms in operation.

Competition has become particularly apparent in the fees institutional investors are paying to invest in hedge funds. The industry standard has long been the so-called two-and-twenty model, in which managers earn 2 percent on assets under management, plus 20 percent of profits returned — arrangements that Berkshire Hathaway CEO and perennial hedge fund critic Warren Buffett recently said “eat up capital like crazy.”

But fees have come down. Institutional investors have managed to negotiate management fees to around 1 percent and profit sharing fees of 15 percent. JPMorgan’s Capital Introduction Group calculated that management fees fell for nearly every type of hedge fund in 2015, averaging between 1.6 percent and 1.7 percent.

Meanwhile, some institutional investors have decided that hedge funds just aren't worth the trouble. The most recent big money pool to publicly pull back was the New York City Employee Retirement System, the city’s largest pension fund, which announced last month it was liquidating its hedge fund portfolio, citing big fees and bad performance.

Blackstone CEO Steve Schwarzman has predicted those outflows will continue. “The rates of return in the hedge fund business and the fee structure have discouraged some investors,” he said. “The asset class will shrink a bit.”

Of course, high-profile retreats from hedge fund investments don’t spell the end of the industry. Top-performing firms still command high fees, and institutional investors still clamor for the size of returns that hedge funds promise, and occasionally deliver. “We’re getting really good-sized flows,” Schwarzman said, noting that it’s the smaller funds that have been washing out.

But a tough market hasn't been the only headache for hedge fund managers. Across the street from the Sohn conference held Wednesday at New York's Lincoln Center, an activist group known as the Hedge Clippers held signs and marched.

Glenview Capital Management LLC | FindTheCompany

“Hey, hedge funds, you can't hide! We can see your dirty side!”

The speculators in attendance couldn't hear the chants. But in the arena of public opinion, anti-hedge fund fervor has picked up, particularly as the presidential election season has ramped up.

Democratic presidential contender Sen. Bernie Sanders has pilloried hedge funds as part of his larger critique of economic inequality in the U.S. And his Democratic rival, former Secretary of State Hillary Clinton, has highlighted the low taxes that some hedge fund managers pay relative to ordinary Americans, promising to close tax loopholes related to capital gains.

Activist groups have added to the pressure. Hedge Clippers has put a spotlight on hedge funds’ investments in oil and gas companies, arguing that these positions clash with the Sohn Conference’s goal of benefiting children's health.

Though that critique isn't likely to sway hedge funders, it adds to negative public sentiment, giving pension managers reason to question the big fees they pay to investment managers. “Hedges have underperformed, costing us millions,” New York City Public Advocate Letitia James said as the city’s pension abandoned hedge funds. “Let them sell their summer homes and jets, and return those fees to their investors.”