North Korean officials were expected to meet Tuesday with their South Korean counterparts in the neutral village of Panmunjom to take back into custody North Korean fishermen who were picked up in South Korean waters earlier this month, according to a new report. Though the North was demanding all five fishermen be repatriated, South Korean authorities said only two have expressed the desire to return to the North.

The five fishermen were found stranded in South Korean waters after their vessel experienced engine failure on July 4. Though Pyongyang has made repeated demands for the return of their citizens, Seoul officials say three of the fishermen have expressed the desire to defect to South Korea.

In an effort to convince the three would-be defectors to return, Pyongyang officials will be accompanied by the family members of those who were considering defection at the Panmunjom meeting. “We strongly urge South Korea to take actions to allow their family members to meet with the fishermen,” North Korea’s Red Cross said Monday in an official memo according to South Korea’s Yonhap News Agency.



North Korea has vowed to take “stern actions” if all five fishermen are not present Tuesday at the meeting.

Still, South Korea has rejected requests from the North, saying only two of the ship men will be at Panmunjom. “We notified the North that Seoul will repatriate the two sailors on Tuesday at the truce village,” South Korea’s Unification Ministry said, citing the wishes of the remaining three to remain in South Korea.

The number of defectors going to South Korea has fallen in recent years as quality of life marginally improves and borders tighten under Kim Jong-Un’s leadership, according to a report by the Seoul’s Unification Ministry. Despite this, South Korea still remains a destination for handfuls of defectors who want to escape the oppressive regime.