Russia is housing an estimated 5,000 at a temporary camp in Bezimenne, east of Mariupol, seen in satellite images.
Ukraine's Deputy Prime Minister Iryna Vereshchuk said 40,000 had been moved from Ukraine to Russian-held territory without any coordination with Kyiv.
A Mariupol refugee, now in Russia, said: "All of us were taken forcibly".
Some Ukrainian officials describe Russia's actions as "deportations" to "filtration camps" - an echo of Russia's war in Chechnya, when thousands of Chechens were brutally interrogated in makeshift camps and many disappeared.
