The sex abuse lawsuit facing Bryan Singer may cost him his spot as director of the upcoming "X-Men" film.

According to Radar Online, a source speaking with the site says Singer and 20th Century Fox are at odds after Michael F. Egan III sued the director in April, accusing him of sexual abuse. Egan later filed sexual abuse lawsuits against entertainment executives Garth Ancier, Gary Goddard and David Neuman. He has since dropped his suit against Neuman. 

The source says that Fox is debating if Singer should direct the next film in the latest "X-Men" trilogy, “X-Men: Apocalypse.” They are specifically worried about the case affecting the project’s schedule.

“Fox isn’t confident in it not heavily impacting production,” the source said. “Bryan’s lawyers are going nuts because the deal was closed months ago.”

Though the source says Singer’s team has offered proof to Fox that the allegations in the lawsuits are false, the studio is still concerned about the financial fallout of the lawsuit.

“They don’t really care whether he’s innocent or guilty -- they’re only concerned with the bottom line, which is giving him another $250 to $350 million to make Apocalypse. There have been talks about who else he’d let direct it, with him producing, but those led to heated arguments and walk outs,” the source said.

The lawsuit going to trial is also a major concern for Fox, says the source. If a trial occurs, both the production and release dates of the film could be pushed back. “They’re also looking at how far back they can push production without moving the release date, if any lawsuit did get a court date,” the source said.

According to the source, Singer still stands to get paid from the deal, even if he doesn’t direct the film. The deal with Fox was reportedly made official several months ago, so he is guaranteed a hefty sum of money for the project since the studio -- if they removed him -- would have made that decision on its own.

In his federal lawsuits against the execs and Singer, Egan claims that the men had sexually abused him when he was a teenager. The 31-year-old says that numerous young boys were sexually abused at a home in Encino, Los Angeles, and that he was twice flown to Hawaii with the men in charge of the Encino house party. He says he was sexually abused by Singer at the L.A. house and in Hawaii, and he was promised movie roles by the director. 

Singer has spoken out against the accusations, with his attorney Marty Singer calling the lawsuit "absurd and defamatory" and the claims "completely fabricated."

"We look forward to our bringing a claim for malicious prosecution against Mr. Egan and his attorney after we prevail," said Marty Singer in an email to the Los Angeles Times.

“X-Men: Days of Future Past,” which Singer directed, has passed $600 million in sales at the worldwide box office after its May 23 release, reports the Wrap.