The yen was 2.3 percent weaker against the dollar compared with its year-before level, which helped push up imports costs.
That missed forecasts for a surplus of 43.3 billion yen following the 481.1 billion yen surplus in April.
Analysts expected exports to rise 16.1% and for imports to increase 14.8%.
May data showed "a correction in speed" but the Japanese economy is likely to continue to expand in the July-September quarter even if it stalls in April-June, he said.More news: 'Significant progress' made in talks between Foster & May
A large trade surplus could draw criticism from the Trump administration, which has repeatedly indicated that it prefers protectionist policies to reduce the US trade deficit and increase exports.
Exports rose at an annualized 14.9% in May, following a gain of 7.5% month before.
Japan logged a surprise deficit of 203 billion yen (1.8 billion), according to data from the finance ministry, despite market expectations that the world's third largest economy would have posted a surplus.
Imports climbed an annual 17.8% to 6.054 trillion yen versus expectations for 14.5% and up from 15.2% a month earlier.
"You can say domestic demand is doing well, but this is being driven more by the manufacturing sector", said Hidenobu Tokuda, senior economist at Mizuho Research Institute.