Two victims of Bernard Madoff's Ponzi scheme filed a federal lawsuit against the U.S. Securities and Exchange Commission on Wednesday, seeking at least the $2.4 million they lost in the fraud.

The victims, believed to be the first to sue the SEC over the $65 billion fraud, said in court papers that the agency was negligent and breached its duties by failing to investigate Madoff despite numerous warnings and tips.

The suit, in New York federal court, is likely to be closely watched by other Madoff victims who blame the SEC for failing to detect the scam.

It is being brought by Phyllis Molchatsky of New City, New York, who lost $1.7 million, according to court documents filed by the Herrick Feinstein law firm. The other plaintiff is Steven Schneider, a New York doctor, who lost about $750,000.

The suit said that while Madoff, who is serving life in prison, is the chief culprit in the scheme, the SEC must be held accountable for its negligence and failure to expose the fraud.

The SEC had countless opportunities to stop the Ponzi scheme Madoff operated over 16 years, and botched all of them, said the complaint, which draws heavily from a report by the SEC's Office of Inspector General.

The agency did not immediately comment.

Other victims have brought suits against Madoff. A court-appointed trustee winding down Bernard L. Madoff Securities Investment LLC has filed suits seeking $15 billion, including $243 million from the Madoff family.

The trustee has recovered $1.5 billion so far under the auspices of the Securities Investor Protection Corp.

Legal experts have said the case is novel because the doctrine of sovereign immunity historically has limited the types of cases that could be brought against the SEC and other federal agencies.

Molchatsky and Schneider submitted administrative claims to the SEC under the Federal Tort Claims Act. After the SEC rejected those claims, the suit said, the two had the right to sue in federal court.

Howard Elisofon, a partner at Herrick Feinstein, has said he expects to file similar administrative actions against the SEC on behalf of other Madoff victims.

The case is Phyllis Molchatsky and Steven Schneider v United States of America, U.S. District Court, Southern District of New York.

(Additional reporting by Grant McCool and Martha Graybow in New York)