American Apparel owner Dov Charney
American Apparel founder Dov Charney was fired by the board for misusing company resources. In the picture: Dov Charney speaks during a May Day rally protest march for immigrant rights, in downtown Los Angeles May 1, 2009. Reuters

Dov Charney, founder and former CEO of American Apparel Inc (NYSEMKT:APP), who was dismissed from his position last week over a 2011 “sex slavery” case, will take legal action against the retailer, claiming that he was wrongfully terminated by the board, according to the New York Post.

Charney, who had refused to step down from his position, was fired Wednesday after the board learned that he allegedly allowed an employee to publish nude pictures of a former company employee, Irene Morales, in 2011, the Post reported Sunday. But, Charney has denied the allegations stating that he was not aware of the blog before it was published.

American Apparel reportedly said that the termination "grew out of an ongoing investigation into alleged misconduct," and the company's board also claimed that Charney had handled company funds in an improper manner.

Charney's attorney, Patricia Glaser, reportedly sent a letter to the board’s council accusing the board of acting illegally and demanded that Charney be “fully reinstated to his positions” and called for a meeting with the board by Monday. The letter reportedly claimed that American Apparel "violated its legal and contractual obligations to Mr. Charney in numerous respects" that have resulted in "substantial professional, reputational and financial injuries to Mr. Charney."

Glaser also reportedly wrote in the letter that the allegations against the 45-year-old Canadian businessman, who founded the Los Angeles based-clothing company, were “baseless” and “most involve activities that occurred long ago (if at all) and about which the Board and the Company have had knowledge for years.”

She also claimed that Charney’s employment contract was renewed by the board in 2012, more than a year after the controversy surfaced.

In a statement to USA Today about Glaser's letter, Allan Mayer, co-chairman of American Apparel, said: "It is exactly what we would expect to get from Dov's attorney in a situation like this. We continue to believe we did the right thing for the right reasons in the right way. We are very confident we are on very strong legal grounds."

Charney has been surrounded by a series of sexual harassment allegations over the past few years but the cases were either settled or dismissed. Morales, who alleged that Charney forced her to perform sexual acts, sued him for $260 million in 2011, but, the case was dismissed in 2012.

Mayer told CNNMoney that he thinks firing Charney will ultimately help the retailer.

"There are people who will tell you that Dov's reputation was a real drag on the company. There were people and firms unwilling to do business because Dov ran the company," said Mayer. "Since we announced the decision, we've been contacted by mainstream, top of the line, institutions that have not been interested in supporting us."