The blast killed at least 135 people and injured more than 4000 others. A two-week state of emergency has begun.
President Michel Aoun said the blast was caused by 2750 tonnes of ammonium nitrate stored unsafely in a warehouse.
Customs chief Badri Daher said his agency called for the chemical to be removed, but "this did not happen".
"We leave it to the experts to determine the reasons," he said.
Ammonium nitrate is used as a fertiliser in agriculture and as an explosive.
Opening an emergency cabinet meeting on Wednesday, President Aoun said: "No words can describe the horror that has hit Beirut last night, turning it into a disaster-stricken city".
What triggered the explosion?
The ammonium nitrate had reportedly been in a warehouse in Beirut port for six years after it was unloaded from a ship impounded in 2013.
The head of Beirut port and the head of the customs authority both told local media that they had written to the judiciary several times asking that the chemical be exported or sold on to ensure port safety.
Port general manager Hassan Koraytem told OTV that they had been aware that the material was dangerous when a court first ordered it stored in the warehouse, "but not to this degree".
Lebanon's Supreme Defence Council has vowed that those found responsible will face the "maximum punishment" possible.
Economy Minister Raoul Nehme told the BBC: "I think it is incompetence and really bad management and there are a lot of responsibilities from management and probably previous governments. We do not intend after such an explosion to stay silent on who is responsible for what."
Putting port officials under house arrest is an attempt by the government to show the Lebanese people that it is serious about enforcing accountability, the BBC's Arab affairs editor Sebastian Usher says.
But many will dismiss it, as they effectively accuse the entire political elite of culpability through their alleged negligence, incompetence and corruption, Usher adds.
The ammonium nitrate arrived on a Moldovan-flagged ship, the Rhosus, which entered Beirut port after suffering technical problems during its voyage from Georgia to Mozambique, according to Shiparrested.com, which deals with shipping-related legal cases.
The Rhosus was inspected, banned from leaving and was shortly afterwards abandoned by its owners, sparking several legal claims. Its cargo was stored in a port warehouse for safety reasons, the report said.