Britain's Boris Johnson is known for his colourful love life but the accusation he grabbed a young woman's thigh when he was a magazine editor risks a scandal as he seeks to unite his party over Brexit.

The prime minister, who is attending his Conservative party's conference in Manchester, northwest England, is accused of giving a female journalist's leg "a squeeze" while at a private lunch 20 years ago.

Johnson never comments on his personal life but his spokesman issued a clear denial -- an acknowledgement that the claim is more serious than gossip about his affairs.

It follows accusations that Johnson gave special access to a US businesswoman with whom he is alleged to have had a relationship when he was mayor of London.

He denies any impropriety, but it is yet another headache after his defeats in parliament and the courts over his pledge to leave the EU on October 31, come what may.

Powerful men

In an article published in The Sunday Times, journalist Charlotte Edwardes described a boozy lunch in the offices of The Spectator magazine around two decades ago, when Johnson was editor.

She was sitting next to him and wrote that "under the table I feel Johnson's hand on my thigh. He gives it a squeeze".

"His hand is high up my leg and he has enough inner flesh beneath his fingers to make me sit suddenly upright."

She said a woman she spoke with after the dinner, who was sitting on Johnson's other side, said the same thing happened to her.

Johnson was married at the time to his second wife. They are now divorced and he currently lives with his girlfriend Carrie Symonds.

Downing Street said Edwardes' claim was "untrue", and the prime minister himself denied it in a television interview, while refusing to comment further.

But health minister Matt Hancock said he knew Edwardes well, and "I entirely trust what she has to say".

Former Conservative minister Justine Greening, who is now an independent MP, said the claim was "deeply concerning".

It went "to the heart of this question about character and integrity of people in public life and what standards the electorate have a right to expect," she told BBC radio.

The allegation comes two years after defence minister Michael Fallon quit following revelations he tried to kiss a journalist.

Opposition Labour lawmaker Dawn Butler added: "What is it about powerful men feeling entitled to harass women? Boris Johnson has serious questions to answer."

A feminist

Yet at the party conference in Manchester, many delegates were sanguine -- while a number of female Conservatives offered their support for Johnson.

"He's a decent person. I think he cares a great deal about women and girls," former defence minister Penny Mordaunt told a fringe meeting.

MP Rachel Maclean highlighted Johnson's campaigning against female genital mutilation, declaring him "a feminist".

Fleur Butler, national chairman of the Conservative Women's Organisation, told AFP the claim related to a time when "the world was a different place".

She said she did not know if the claim was true but added: "It's really important though for the party to say that we do not accept this type of behaviour moving forward."

Brexit first

Johnson is no stranger to personal scandal, most recently over a noisy row with Symonds that resulted in police being called to their flat in south London.

There are concerns that the latest claims could damage Johnson's standing with women, who are already less likely to support him.

A YouGov survey this week found 44 percent of men thought Johnson was likeable, compared to 34 percent of women.

One female Conservative member in her 20s, speaking on condition of anonymity, told AFP: "I think it's a really big issue.

"There are still some older members of the Conservative party who don't treat women as they should."

But for many in Manchester, the biggest issue remains Brexit -- and they believe Johnson is the man who can deliver that, regardless of anything else.

"We need to get somebody to get Brexit done," Margaret Dane, a retired Tory party member, told AFP.

As for the squeezing allegation, she added: "I'll just have to discount the fact that it might be the truth."

Others dismissed it as yet another story "dredged up" to discredit the prime minister.

"It should be looked into. But so much is coming out, I think people are getting immune to it," said 48-year-old councillor Julian Goodrich.