Protesters in a small Serbian border town called for government action on Sunday over the town's rapidly rising population of asylum seekers from Afghanistan, the Middle East and Africa trying to reach the European Union.

Banja Koviljaca, a town of around 6,000 people on the river Drina between Serbia and Bosnia, is currently home to an estimated 2,500 migrants sent to an asylum centre there after being stopped by police trying to reach Western Europe.

The asylum centre, one of only two in Serbia, has only around 80 beds.

Anger has been rising since the arrest of several Afghan men accused of raping a British woman in the town last month.

Serbia has seen a sharp increase in recent years in the number of asylum seekers, and the figures are expected to rise further as the Balkan country -- bordering EU members Romania, Bulgaria and Hungary -- moves towards membership of the EU.

Speakers at a protest of around 500 people said the town's children would not go to school until the government solves the issue.

At first dark, people lock their houses, people are scared, local resident Natalia Maksimovic told the crowd.

Yesterday it was rape, tomorrow it will be murder, while we stay silent and wait for someone who doesn't even know where Banja Koviljaca is to solve the problem.

In the first six months of this year, the United Nations High Commission for Refugees (UNHCR) registered a five-fold increase in the number of asylum seekers in Serbia, including from Afghanistan, Somalia, Pakistan and the Middle East.

None could be seen on the streets of Banja Koviljaca on Sunday during the protest, but riot police were present amid warnings from rights groups of a deepening mood of animosity.

Serbian Interior Minister Ivica Dacic has raised the possibility of opening other asylum centres.

They are in the country in line with international conventions and we can't just expel them, he said last week, Serbian media reported.

Serbian citizens themselves account for a high number of asylum seekers in rich countries. Taken together with its former breakaway Kosovo province, Serbia ranks third behind Afghanistan and China in terms of asylum seekers trying to reach the West, according to U.N. statistics.

(Writing by Matt Robinson; Editing by Peter Graff)