Rob Kardashian has become a recluse over the years, which led him to avoid Caitlyn Jenner and the rest of his siblings at her Father’s Day bash in her Malibu, California, mansion Sunday. The only Kardashian brother didn’t attend the party because he was uncomfortable with his weight, an insider told Radar Online in an exclusive report Monday.

Though he declined to join Jenner for her celebration, he still sent a special message. “Rob decided not to go to Caitlyn’s house because he just didn’t feel like being around a lot of people,” an insider told Radar Online. “He communicated with Caitlyn during the day, and wished her a Happy Father’s Day."

Kardashian, 28, faced a major backlash after photos of him outside a fast-food restaurant in Los Angeles surfaced. He was wearing an oversized black T-shirt and black pants, his standard attire since he gained at least 50 pounds. “He just isn’t in a good place and hasn’t been for a very long time. Food is his comfort,” a source said.

The sock designer is reportedly coping with depression and has been unable to fight the battle of the bulge. "He's still struggling," a source told People magazine, adding that Kardashian "just can't get motivated. Everyone is trying to be encouraging, but for the most part, he doesn't want to hear what he already knows."

He has missed out on countless family events, as documented in the family reality show “Keeping Up With the Kardashians.” "He's sad, he's bitter, he's single," a source told People back in March. "He is very depressed and has been [for a while]. It's really hard for him that his sisters are so public and he looks like the shut-in." 

For all the latest Kardashian drama, don’t miss “KUWTK” on E! every Sunday at 9 p.m. EDT.

Follow me on Twitter @mariamzzarella