The France worldwide, who replaced compatriot Alexandre Lacazette in the 72nd minute of a lively game, scored two minutes from fulltime with a glancing header off an Alexis Sanchez cross.
The Saints took the lead in just the third minute, when Dusan Tadic burst forward from the throw-in and played a delicious reverse pass for Charlie Austin to run onto and thump home the opener past Petr Cech.
For Giroud however, he can celebrate matching a record set by Manchester United legend Ole Gunnar Solskjaer.
"He is mentally extremely strong and you have to be grateful that a player, whenever he is needed, comes on and delivers. It's a remarkable achievement what he has done again today".More news: Comedian Hannibal Buress arrested for disorderly intoxication in Miami
The French global, who has 69 worldwide caps under his belt, was heavily linked with a move to Everton in the summer transfer window. "Should have got the second really", said Austin, who had a low shot from the right well saved by Cech minutes after scoring.
"Against this team [Arsenal] you have to play close to excellent, because they don't need too many actions to create problems but in both aspects, attacking and defending, we were intelligent and focused from beginning until the end", Pellegrino added.
Sutton also criticised Arsenal's defence. Southampton were still risky against the run of play, however, with Spanish defensive midfielder Oriol Romeu firing in a long-range shot that smacked the crossbar in the 62nd. A back 3 of Laurent Koscielny, Nacho Monreal, and Per Mertesacker (who replaced injured Shkodran Mustafi) struggled at the start of the game, and the team sure paid the price for it.