Dastyari and the Gellibrand MP, Tim Watts, were in the Victoria University student bar on Wednesday evening when a group of far-right nationalists intruded.
Video of men identifying themselves as "working class patriots" heckling the Iranian-born senator, calling him a "terrorist" and "little monkey" at the VU Bar in Footscray was posted to the Patriot Blue Facebook group last night.
He admitted that it makes him feel small, disgusting and "feel kind of bad and that's what they are created to do".
For Dastyari, the verbal assault was "the sickening face of white nationalists".
"I worry more about the 15-year-old girl getting the school bus home today having to get this kind of abuse than a politician that's lucky enough to have the some support structures around him".
"In my time, did I throw it back to anyone, no I didn't, I stood my ground for what I believed in and I think he is just using this to get publicity to sell his book, I don't see anything in this, I really don't", she said from Queensland.
Dastyari says he is consulting with his family before deciding whether to lodge a formal complaint against his abusers.
Prime Minister Malcolm Turnbull has commented on the incident, saying there should be "zero tolerance" for racial abuse.
The senator, who posted the video of the incident on Facebook, has warned of a rise in white nationalism after the incident.More news: The DOJ may be pushing AT&T to sell off CNN
One of the men who was convicted, Neil Erikson, was among the group who abused Dastyari on Wednesday.
Oh, and we nearly forgot, a very special mention goes to Labor MP Tim Watts for his handling of his colleague being racially abused.
Instead, she blamed Dastyari and called him "a little Mr Bean" and "a smartarse", The Guardian reports.
Labor leader Bill Shorten also said he was disgusted by the "ugly" incident. "What is this country coming to when you can not go out for a meal without being abused by racist idiots", he told Seven.
"I think he's a smart ass, he shows it on the floor of the parliament, and he plays up his position (as a Muslim)".
In a statement Toll said "the individual wearing the Toll polo shirt is not employed by Toll, and has not worked for us for several months".
A Toll spokesperson told AAP the actions in the video "in no way reflect our beliefs and values as a company".
"Freedom of speech is a fundamental right but it should never be used to justify inciting hatred that risks causing harm to people", she said. "If he wants to go for Toll and try and hurt me financially he can go for it".
He's also appealing a 2017 conviction for beheading a dummy outside a Bendigo council office in protest over plans to build a mosque.