Police said at least 20 fighters from the Maute group had turned up at the Lanao del Sur jail in the southern city of Marawi, but no shots had been fired.
A military source told the BBC they believed the men had been allowed to escape.
The militants were held last week after being caught with homemade mortars.
At least 15 other inmates - who faced murder and drugs charges - also walked free, but it is not clear whether this was agreed.
The Maute group has carried out several bombings and kidnappings in the southern Mindanao region.
The Philippines has faced separatist movements for decades in Mindanao, which is majority Muslim - the rest of the Philippines is mainly Catholic.
The Maute group carries the black flag and insignia of IS, and has attacked army troops, beheaded a soldier and beheaded two local workers earlier this year.
The militants kidnapped the two workers and made them wear orange shirts similar to those worn by IS beheading victims before they were killed.
Several armed groups in the Philippines have pledged allegiance to IS, although the country's military says there is no evidence of active co-operation with foreign militants.