Magufuli's win in the October 25 poll with over 58% of votes cemented the long-running Chama Cha Mapinduzi (CCM) party's firm grip on power.
Vice-President Samia Suluhu Hassan, who comes from Zanzibar, was also sworn in at a stadium in the economic capital Dar es Salaam, an AFP reporter at the ceremony said.
Elections were largely peaceful, but the opposition said the vote was rigged and also claimed victory, while semi-autonomous Zanzibar -- which also voted for its own president -- annulled polls over irregularities.
Outgoing President Jakaya Kikwete has said he is "so happy" to be leaving his job after a decade in power, having stepped aside after serving his two-term limit.
Magufuli, a 56-year old former chemistry teacher, ran on an anti-corruption platform, securing a convincing victory over his closest rival, ex prime minister Edward Lowassa who won 40%.
Those at the ceremony included African Union chairman, Zimbabwean President Robert Mugabe, as well as regional leaders including Kenya's Uhuru Kenyatta, Uganda's Yoweri Museveni and Rwanda's Paul Kagame.
Other neighbouring leaders included Filipe Nyusi of Mozambique and Zambia's Edgar Lungu. South Africa's Jacob Zuma was also present.
On Zanzibar, opposition supporters on the Indian Ocean archipelago said they were "unhappy" that the mainland had pressed ahead with the swearing in ceremony.
Zanzibar's electoral commission ruled last week that the October 25 vote on the islands -- where the 500,000 registered voters also cast ballots for Tanzania's national president -- must be carried out again, citing "violations" of electoral law.
The annulment came after a key candidate, Seif Sharif Hamad of the opposition Civic United Front (CUF), declared himself the winner before the results were officially announced.
Homemade bombs exploded in Zanzibar town over the weekend. No one was wounded, but the explosions sparked concern on the islands, whose economy is dependent on foreign tourists.