Indian Prime Minister Narendra Modi criticized the so-called cow vigilantes Saturday, saying such people made him “angry.”

Self-appointed cow vigilantes have fashioned themselves as protectors of cows, an animal which is considered sacred by many Hindus. These cow vigilantes were responsible for the public flogging of four Dalit men (Dalits are the lowest of the four castes in India’s traditional caste system) for skinning a dead cow in July. The incident took place in Modi’s home state of Gujarat in eastern India. In a separate incident, vigilantes attacked two Muslim women who were allegedly carrying beef in Madhya Pradesh state in central India.

The prime minister has been under fire for not commenting on such incidents. Reports said that Modi may have remained silent in order to maintain his appeal among Hindu nationalists.

Cow vigilante group Members of a cow vigilante group go out on a patrol in the hopes of stopping vehicles of cow smugglers in Ramgarh, Rajasthan, India, Nov. 8, 2015. Photo: ALLISON JOYCE/GETTY IMAGES

But in his first-ever town hall-style address Saturday in New Delhi, the prime minister reportedly said, “I get so angry at those who are into the Gau-Rakshak (cow vigilantes) business. A Gau-Bhakt (cow devotee) is different, Gau Seva  (cow protection) is different. I have seen that some people are into crimes all night and wear the garb of Gau Rakshaks in the day.”

He added that he would urge state governments to prepare a dossier on cow vigilantes. He also said that “70-80 percent [of cow vigilantes] will be those who indulge in anti-social activities and try to hide their sins by pretending to be Gau Rakshaks. If they are true protectors, they should realize that most cows die because of plastic, not slaughter. They should stop cows from eating plastic.”

The prime minister said that nearly two buckets worth of plastic was removed from the stomach of one of the cows when he was working at a health camp for cows.

This is the first time the prime minister has spoken out on cow vigilantism.

Cow vigilantes unload animals Members of a cow vigilante group unload cows to a rescue shelter from a truck they chased down in Ramgarh, Rajasthan, India, Nov. 8, 2015. Photo: ALLISON JOYCE/GETTY IMAGES

India is home to nearly 1.2 billion Hindus with large Muslim, Christian and Buddhist minorities. Hindu hardliners have been pressurizing Modi’s Bharatiya Janata Party to step up its efforts to protect cows in the country. Opposition parties accused the ruling BJP of encouraging or ignoring this sort of vigilantism which led to the lynching of a Dalit man in India’s Uttar Pradesh state last year. The man was accused of possessing beef.

Consumption of beef has been banned in several states in the country where the lower castes, nearly 25 percent of the total population, are beef eaters.

“Beef is one of the most affordable sources of protein for the Dalit community,” Mohan Dharavath, president of the Dalit Adivasi Bahujan and Minority Students’ Association, told Al Jazeera.