It appears that not everyone in Kate Middleton’s family is a fan of her royal title. While the Duchess of Cambridge’s sister, Pippa Middleton, 31, has settled into a life of fame in recent years, her brother, James Middleton, is having a harder time adjusting. In an interview with TYD (The Young Director) magazine, he admits that having Kate, 33, as a sister can be “frustrating.”

According to James, 27, being known as Kate’s brother instead of a “serious businessman” is exasperating. “Yes, it does get frustrating,” James, an entrepreneur told the digital magazine in its winter 2015 issue (via the New York Daily News). “I work incredibly hard, just like every other person.”

James, founder of the Cake Kit Company, said he is focusing on other projects. His most recent business venture is Boomf, a company that prints Instagram photos onto gourmet marshmallows. On its website, the business — James serves as product director — says it creates “the edible selfie.”

While his relationship with Kate doesn’t appear to be affecting his business, it’s not just his association with the duchess that upsets the Reading, England, native. He also said it's difficult for people to accept him as a normal person. “And aside from the fact— yes, I am the brother of someone very important — I am, at the end of the day, just James,” he said.

In the tell-all interview, James also discussed his childhood. Despite having a privileged upbringing — his parents, Carole and Michael Middleton, are founders of the party supply company Party Pieces — he did face personal struggles. He revealed that he suffers from dyslexia, which caused him to drop out of the University of Edinburgh. As an adult, James said, his dyslexia is now one of his “strong points.”

“For one, it makes you unique,” he said. “And two, your brain works differently. You will get to the answers of a solution, but just in a different, maybe unconventional way."

Unfortunately for James, it doesn’t appear his family will be out of the public eye for quite some time. Not only did the Middletons recently gain attention for their trip to the private island of Mustique for Carole’s 60th birthday, they’re also bound to be brought into the spotlight again following the birth of Kate and Prince William’s child in April. The baby will be the royal couple's second. They welcomed their first, Prince George, in 2013.