Members of a South Carolina-based Ku Klux Klan (KKK) group called the Loyal White Knights showed up to Charlottesville, Virginia to protest the removal of a statue of Confederate General Robert E. Lee on Saturday. The group had around 30 to 50 members, according to NPR Sunday.

The arrival of the white supremacy group spurred on a large counter protest. According to CNN Saturday, there were around 1,000 people there shouting at the KKK with chants like “racists go home.”

READ: Confederate Monuments Removed: Protesters Clash Over New Orleans Statues

At one point the counterprotesters surrounded the members of the KKK and wouldn’t allow them to leave. The police then had to step in and escort the KKK back to their cars. The police declared an unlawful assembly and tried to disperse the crowds. The police had to revert to using tear gas and pepper spray. 

“There were a number of incidents, including the use of pepper spray by the crowd,” said Miriam Dickler, a city spokeswoman.

Over the course of the day,  23 people were arrested.

“They're trying to erase our history, and it's not right what they're doing,” said Klansman Douglas Barker to NBC’s Charlottesville associate WVIR. “We believe in preserving our country's heritage. We don't go into other countries and take down their monuments.”

Charlottesville Mayor Mike Signer told residents to avoid the protest on his Facebook.

“Throughout the day, we urged folks not to take the bait -- to deny the KKK the confrontation and celebrity they desire," Signer said. “Thousands of people followed that advice, attending a wonderful set of alternative events.”

Charlottesville’s city council voted to remove the statue in March and change the name of the park where the statue was located from Lee Park to Emancipation Park, according to CNN Saturday.

“Charlottesville has kind of been put on the map recently," said Signer. “We want to change the narrative by telling the true story of race through public spaces. That has made us a target for groups that hate that change and want to stay in the past, but we will not be intimidated.”

In May the statue was the site of a protest by noted white nationalist Richard Spencer.

READ: Who Is Steven Joshua Dinkle? Ex-KKK Leader Sentenced To Prison In Sexual Assault Case

A court stayed the removal of the statue and handed down a six-month injunction from removing the statue because of several lawsuits against the action.

Across the South, there have been several dust-ups as cities reckon with their history of slavery and the Civil War. The removal of Confederate statues in New Orleans caused a similar uproar of pro-Confederacy groups.