Speaking at the side of the German leader, Macron said that they need to work on "deep reforms that are necessary and need common work".
He said he will implement a reform agenda at home "not because Europe requests it, but because France needs it".
Macron and his new prime minister face major challenges.
Inside Story - What does France's new president-elect have to offer?
Like Macron, France s new prime minister has little truck with the entrenched left-right divide.
Merkel said at the weekend she wanted close cooperation with Macron and that their two countries would do everything to shape European policy. France's unemployment rate remains stuck at 10 percent, roughly double the level in the United Kingdom and Germany.
Hours ahead of Chancellor Angela Merkel's meeting in Berlin with Macron, her spokesman Steffen Seibert said one should not read too much into the alarmist headlines. It is traditional for French leaders to make Berlin their first European stop.
Many conservatives around Merkel are sceptical of Macron's calls for closer integration.
The visit aims at highlighting the importance of the French relationship with Germany in relaunching the European project.
Alain Juppe, a former French prime minister, called Philippe "a man of great talent" with "all the qualities to handle the hard job".More news: Alonso hopes his first finish boosts McLaren's 2017 season
One of Macron s aides welcomed his appointment as "a good move", telling AFP it would help him "break the right".
"I told myself that the situation we were in was so unique that we should try something that had never been tried before", Philippe said on TF1 television later on Monday.
Philippe praised Cazeneuve's "quite exceptional" ministerial career and his "constant elegance". The main role is to preside over the government and carry out the program of the president.
Her three children from her previous marriage were also present at Sunday's inauguration along with VIPs from France's political scene and the young team of advisors behind Macron's sensational rise.
He was a socialist in his younger days before switching his allegiances to the right.
Melenchon got almost 20 percent of the vote in the first round of the presidential election.
Voters go to polls again in June to elect 577 National Assembly lawmakers.
Philippe, 46, is now mayor of the northwestern French city of Le Havre and a conservative lawmaker since 2010 and will take over from Socialist Prime Minister Bernard Cazeneuve who resigned on May 10 to pave the way for the newcomer.
Meanwhile, Mr Juppe described the new prime minister as "a man of great talent" with "all the qualities to handle the hard job". He accompanied Juppe in the creation of the UMP party (now The Republicans) and served as the party's director of services until 2004. Philippe quit that cause when Fillon's campaign was hit by a financial scandal over publicly funded jobs for members of his family.
Trained as a lawyer, Mr. Philippe has worked in the private sector, including for French nuclear giant Areva and an American law firm.