Slovenian rider Matej Mohoric of the Bahrain-Merida team edged Nico Denz of AG2R in a two-man sprint to win the stage while all of the other leaders finished safely in the main pack, 34 seconds behind.
Come the finish line on Piazza Municipio, Dumoulin trailed Yates by just two seconds, though the Briton also picked up an additional four seconds in time bonuses.
But once Chaves was dropped, the teams of overall hopefuls, including four-time Tour de France victor Chris Froome, reigning champion Tom Dumoulin, local favourite Fabio Aru and French hope Thibaut Pinot, pushed the pace in the peloton to ensure Chaves never regained contact.
"He was chasing me and he looked better than other days on steep finishes".
Bennett headed into the day 1m 36s behind stage victor and pink jersey holder Simon Yates, but he now sits 2m 9s behind the Brit after his progress was stalled after he broke his chain on the final climb and had to change his bike. "Straight after the rest day, you don't know how the body responds". Dumoulin has been consistent but unspectacular when the road has climbed to this point, but his fierce pursuit of Yates at Osimo suggested that his condition is in crescendo as the Giro reaches its midway point.
"It was the most aggressive stage I've seen for a while", said Yates.
"He's getting better as the race goes on".
The four-time Tour de France victor now trails his fellow Briton by three minutes and 20 seconds as his hopes of winning the race continue to fade.More news: The Celtics Have Dominated the Cavs With Depth And Relentlessness
Mohoric thanked his team, which is centered around Domenico Pozzovivo in fourth overall, for letting him go for the victory.
"I'm delighted that I have taken this opportunity", said former UAE Team Emirates rider Mohoric. I already have a good gap to Froome.
Ireland's Sam Bennett won the sprint for third place 36sec later.
What was expected to be a quiet transition stage - over the longest distance of the entire giro at 244km - turned into a frantic day of racing after Chaves was dropped on the category two climb which opened the day's racing.
"It's not a two-way battle, because there are four to five guys who are still close".
Thursday's 12th stage is a flat, 214-kilometer route from Osimo to the motorsport race circuit of Imola.
"It's good to get this racing in the legs". The race ends in Rome on 27 May.