The brother and sons of imprisoned swindler Bernard Madoff are the subject of criminal tax-fraud cases by federal prosecutors in Manhattan, the Wall Street Journal reported, citing people familiar with the matter.

Peter Madoff, the brother, was the compliance officer for Bernard L. Madoff Investment Securities LLC in New York and held an executive post with Madoff Securities International Ltd in London.

Bernard Madoff's sons, Mark and Andrew, helped run the firm's market-making division, which was separate from the investment arm where Madoff perpetrated his multibillion-dollar Ponzi scheme, the WSJ said.

Lawyers for Peter Madoff didn't respond to requests for comment, while Martin Flumenbaum, a lawyer for Mark and Andrew Madoff, said in a statement they had no prior knowledge of Bernard Madoff's crimes and contacted authorities immediately after their father told them of his fraud, the paper said.

The sons continue to cooperate fully with the authorities in their ongoing investigations, the paper quoted Flumenbaum as saying.

Peter Madoff's lawyer Charles Spada said in a letter to U.S. District Court Judge Madeline Cox Arleo last month that his client was the subject of a criminal investigation by U.S. prosecutors.

Bernard Madoff, 71, is serving a 150-year prison sentence in North Carolina after pleading guilty in March last year to a worldwide investment fraud of as much as $65 billion.

Thousands of investors and charities lost money in the Ponzi scheme -- in which early investors are paid with the money of new clients.

The exact nature of the potential tax violations isn't clear, the WSJ added.

Madoff's trustee could not be reached outside business hours by Reuters for comment.

(Reporting by Archana Shankar in Bangalore, Editing by Ian Geoghegan)