The top U.S. spy Sunday revealed details of a secret mission to North Korea earlier this month that led to the release of two U.S. citizens held captive. Director of National Intelligence James Clapper, who arrived in Pyongyang Nov. 8 to negotiate the prisoners’ freedom, said on CBS “Face the Nation” he met with North Korean officials for a “terse” dinner and at one point, he thought the mission to secure the prisoners might go sour.

"I was quite apprehensive because we weren't sure how this was going to play out,” Clapper said. “I personally was not completely confident that we would actually, that they would actually release our two citizens, and so yes, it was apprehensive.”

Clapper’s visit to the secretive communist country came after North Korea contacted the U.S. government and requested a high-ranking diplomat be sent to Pyongyang to discuss the prisoners, U.S. citizens Kenneth Bae and Matthew Todd Miller. Bae and Miller were the last Americans held by Pyongyang following the release of Jeffery Fowle in October.

Bae, a Korean American missionary from Lynnwood, Washington, had been imprisoned in North Korea since 2012. He was serving a 12-year prison sentence for alleged anti-government activities.

Miller, a tourist from Bakersfield, California, was taken into custody that same year and sentenced to six years in prison for reportedly tearing up his passport and demanding asylum.

Clapper said when he arrived in North Korea, he presented a letter from U.S. President Barack Obama that apologized for the two men’s crimes. The letter also stated it would be a positive gesture for Pyongyang to release the two prisoners.

Clapper said North Korean officials seemed frustrated they didn’t get the high-profile talks with U.S. leaders for which they hoped. "I think they were disappointed, frankly, that I didn't have some breakthrough," Clapper said.

After negotiations, Clapper was given 20 minutes to pack his bags before he was taken to a downtown hotel. He said that was when he knew Bae and Miller would be handed over. “We shook hands. I said thank you. We walked out,” he said.