The ancient landmark with its famous leaning minaret was where IS leader Abu Bakr al-Baghdadi proclaimed a "caliphate" in 2014.
However, IS claims that US aircraft destroyed the complex, in a statement issued by its news outlet Amaq.
Iraqi Prime Minister Haider al-Abadi said blowing up the mosque was "an official declaration of defeat" by IS.
Aerial photos show that the mosque and minaret has been largely destroyed.
The Iraqi commander in charge of the offensive to retake Mosul said troops were within 50m of the mosque when IS "committed another historical crime".
A senior US commander in Iraq said IS had destroyed "one of Mosul and Iraq's great treasures".
"This is a crime against the people of Mosul and all of Iraq, and is an example of why this brutal organisation must be annihilated," Maj Gen Joseph Martin said.
The jihadists have destroyed a string of important heritage sites in Iraq and Syria.
The UN has said IS may be holding more than 100,000 people in Mosul as human shields.
Thousands of Iraqi security forces, Kurdish Peshmerga fighters, Sunni Arab tribesmen and Shia militiamen, assisted by US-led coalition warplanes and military advisers, are involved in the offensive against IS, which was launched on 17 October 2016.
The government announced the full "liberation" of eastern Mosul in January 2017. But the west of the city has presented a more difficult challenge, with its narrow, winding streets.
Offensive's 'final chapter'
The Iraqi statement said both the mosque and its distinctive Hadba (hunchback) leaning minaret had been blown up.
Construction began on the mosque in 1172.
A month after IS forces overran Mosul in June 2014, Baghdadi gave a Friday sermon from the pulpit inside the mosque and proclaimed a caliphate - a state governed in accordance with Islamic law, or Sharia, by God's deputy on Earth, or caliph.
It was his first public appearance in many years.
The Old City is the last district under the control of IS within Mosul itself, which was the group's stronghold in Iraq.
On Sunday, commanders announced the start of the "final chapter" of the offensive, with Iraq's counter-terrorism service, army and federal police attacking the Old City from all directions.
The army said it believed there were no more than 300 militants left in Mosul, compared with almost 6000 at the start of the offensive in October.
Earlier this week aircraft dropped leaflets urging civilians to avoid open spaces and to take any opportunity to escape.
The International Committee of the Red Cross (ICRC) warned on Tuesday that the flow of injured civilians out of western Mosul had increased, with victims having wounds from gunshots, shelling and bombs.
Photo: AFP Mosul's Great Mosque of al-Nuri, last week.