A town in Indiana was forced to remove a cross that was placed atop the town square’s Christmas tree after the American Civil Liberties Union (ACLU) filed a lawsuit Sunday against the town for violating the Frist Amendment. Local officials in Kingston, Indiana, located about 40 miles from Indianapolis, issued a statement Monday after the ACLU sued the town for displaying the “Latin” style cross in the town’s central square, the IndyStar reported.

The suit was brought to the ACLU from Joseph Tomkins, a Kingston resident who complained the public cross violated the First Amendment and he was against his tax dollars going to the upkeep and lighting of the religious decoration. According to the complaint, no other holiday decorations were displayed in the square.

“The cross is the best known symbol of Christianity and Kingstown’s prominent display of this symbol represents an establishment of religion in violation of the First Amendment of the United States Constitution,” read the complaint, according to the IndyStar.

According to the First Amendment, the Establishment clause states that “Congress shall make no law respecting an establishment of religion,” which has been used, most notably, to ensure that public institutions like school cannot promote religious practices or beliefs.

The town of Kingston relented to the complaint, citing that it was not prepared to “pay the legal fees of the ACLU lawyer and monetary damage to the plaintiff,” according to a statement, which was issued by the Kingstown Town Council on the city’s news and event Facebook page.

“It is with regret and sadness that the Kingstown Town Council has had the cross removed from the Christmas tree on the town square and is expected to approve a resolution at the next council meeting stating they will not return the cross to the tree. We could not win the court case brought by the ACLU.”