David Goldman has been involved in a bitter custody battle with the boy's Brazilian relatives, after his ex-wife died.
Father and son were reunited at the US consulate in Rio de Janeiro today at 9 a.m. local time.
This is a very difficult moment, said Sean's maternal grandmother, Silvana Bianchi, according to BBC news.
Goldman's US lawyer, Patricia Apy, was critical of how the handover was carried out.
Unfortunately, the Brazilian family, rather than have the hand-off take place in a garage, which would have been secure, parked away and walked him through the press, which only serves to make the situation more stressful for the child, she told the Associated Press.
Meanwhile, Secretary of State Hillary Clinton said she is thrilled that David Goldman has been reunited with his son and sent them warm wishes for the holidays.
I am thrilled that nine-year-old Sean Goldman was reunited with his father David Goldman earlier today in Rio de Janeiro and that they are flying home to New Jersey, Clinton said in a statement.
I offer my warmest wishes for father and son as they celebrate their first holiday season together in five years, Clinton added.
The international custody battle drew widespread attention after Sean's mother, Bruna Bianchi, took him to Brazil back in 2004. Bianchi told David that she would return in a couple of weeks, but she never did, and instead got a divorce and remarried.
In 2008, she died during childbirth. David Goldman, with the help of NJ Representative Chris Smith, has been fighting to regain custody of his son all these years.