Customs officials said hospital waste was discovered in many of the 263 containers imported by a private firm.
The shipment was meant to be made up of used mattresses, carpets and rugs for potential recycling.
Most of the containers have been stored in warehouses, with only a small amount of material having been re-exported.
Legal action was taken after the Sri Lanka authorities impounded the material in 2018.
Officials said the 21 containers had left Sri Lanka on Saturday.
