Rosenstein's testimony comes amid public statements from Trump confidants that the American president is mulling removing Mueller from the investigation.
Mr Trump can not directly dismiss Mr Mueller.
New Hampshire Sen. Jeanne Shaheen, a Democrat, opened her line of questioning, in the hearing that was supposed to focus on the Justice Department budget, by asking whether Rosenstein had seen any evidence of good cause to fire Mueller.
"I think he's considering perhaps terminating the special counsel", Ruddy said in an interview with Judy Woodruff of "PBS NewsHour". Consequently, he recused himself from any investigations into Russian Federation in March. The White House finally said Tuesday evening that Trump has "no intention" of firing Mueller, amid speculation and concern that he was weighing that option.
He called because "I have been very clear" about Mueller and the lawyers he's hired amounting to a "rigged game", Gingrich said in an interview Tuesday on CBS.
Even so, Gingrich said, Trump won't seek to dismiss Mueller because he's confident the probe will conclude he's done nothing wrong.
Mr Rosenstein is charged with Mr Mueller's fate because Attorney General Jeff Sessions recused himself from all matters having to do with the Trump-Russia investigation.
For many, the prospect of Mueller's firing immediately brought to mind the 1973 dismissal of special prosecutor Archibald Cox during the Watergate scandal.
The Department of Justice's (DOJ) own regulations confirm a conflict of interest by declaring that no one in the DOJ can "participate in a criminal investigation" if he/she has a "personal" relationship with someone substantially involved in the subject of the investigation.More news: Top Tories plot revolt, party tells May her days are numbered
Asked if he had confidence in Robert Mueller, the special counsel appointed to lead the Russian Federation investigation, Mr Sessions replied: "I have confidence in Mr Mueller but I am not going to discuss any hypotheticals", noting that he was not involved in the Russian Federation investigation. Such a move would create a firestorm coming weeks after Trump fired FBI Director James Comey. Mr Gingrich said he is troubled by Democratic donations of Mr Mueller's picks to help lead the probe. "I have confidence in Bob Mueller".
However, White House sources told The Times that Trump is hard to predict, and they can not be sure he won't fire Mueller.
They say Mr Trump did not collude with Russian Federation and see the investigation as a politically motivated sham that handicaps Mr Trump's ability to execute his agenda, according to one person who advises the White House on how to handle the probe. The person spoke on condition of anonymity to discuss private conversations. Susan Collins, R-Maine, Rosenstein said, "Senator, I'm not going to follow any order unless I believe those are lawful and appropriate orders".
Mr Ruddy was at the White House on Monday to meet with White House aides, but did not speak with the president, Press Secretary Sean Spicer said.
Gingrich and Ruddy said it would be a "mistake" for Trump to remove Mueller, although both said they see the probe as unnecessary.
"When the day comes that our country can not rely on people to deliver the truth about whatever the subject is from positions of great authority and great reputations, like Mueller, and if we just dismiss those as not being not able to deliver us the truth about an issue we've got more problems than we think we have", Womack said.
White House spokeswoman Sarah Huckabee Sanders said: "Chris speaks for himself".
Peter Carr, a spokesman for Mueller, declined to comment on Ruddy's remarks.