As the Supreme Court heard arguments Wednesday about the constitutionality of an affirmative action program at the University of Texas, Justice Antonin Scalia suggested admissions plans that consider race harm minority students by allowing them to attend schools that are too challenging for them.

Scalia, a known critic of affirmative action, referenced an amicus curiae brief when making his argument, and said “that it does not benefit African-Americans,” Talking Points Memo reported. He added that those students would be better off at the less challenging schools they might have attended without affirmative action.

The justice went on to argue that “most of the black scientists in this country do not come from the most advanced schools” and they benefit from a “slower track.”

“They're being pushed into schools that are too advanced for them,” Scalia said.

Yahoo News reporter Liz Goodwin tweeted Scalia’s quote and pointed to an amicus curiae brief filed in the University of Texas case that she said she believed Scalia had been citing. The brief, filed by two members of the United States Commission on Civil Rights, includes studies that show African-American and Hispanic students are just as likely to be interested in science or engineering majors as their white peers, but minority students often switch out of these majors before graduating.

“Significantly, African-American and Hispanic students jump ship at much higher rates than whites,” the brief reads. “A recent study at Duke University, for example, found that approximately fifty-four percent of black males switched out of such majors, whereas less than eight percent of white males did.”

In one study cited in the brief, the most important cause for minority attrition from science was the “relatively low preparation of black aspirants to science in these schools.”

One of the people who filed the amicus brief, University of San Diego law professor Gail Heriot, wrote an op-ed in the Wall Street Journal in October arguing that this data supported the idea that affirmative action was harming minority students.

The case currently before the Supreme Court, Fisher v. University of Texas at Austin, was brought by a white woman who was not accepted by the school in 2008 and who says its policy of using race as a factor in admissions is unconstitutional.