At least 10 people were killed and 35 hurt when the opera house, concert hall and government offices were hit in Freedom Square in Kharkiv.
Speaking to the European Parliament, Mr Zelensky urged the EU to prove it was with Ukraine.
Later the main TV tower in the capital Kyiv was hit, putting media off air.
Footage on social media showed smoke billowing from the steel structure.
Five people died in the attack, Ukrainian officials said, but the tower remains standing.
A nearby memorial to victims of the Holocaust was damaged in the same strike. The Babyn Yar ravine is Europe's largest mass grave of the Holocaust where between 70,000 and 100,000 people, mostly Jews, were shot by the Nazis.
Mr Zelensky said on Twitter that the attack was "history repeating...".
"What is the point of saying 'never again' for 80 years, if the world stays silent when a bomb drops on the same site of Babyn Yar?" he asked.
Meanwhile new satellite images emerged of a huge Russian military convoy which has been outside Kyiv for several days, amid fears of an all-out assault on the capital.
The convoy includes armoured vehicles, tanks, artillery and logistical vehicles, and is said to be less than 18 miles (30km) from the city.
But a senior US defence official said there has been "no appreciable movement" by the convoy on Tuesday.
There were indications that morale was flagging among Russian troops in general, and some units were surrendering, sometimes without a fight, the official added.
Russia's defence ministry has urged citizens in militarily sensitive areas of Kyiv to leave their homes.
The ministry, quoted by Tass news agency, said the Russian military was not targeting cities, only military infrastructure, and that there was no threat to the civilian population.
'Terror against Ukraine'
In a speech for which MEPs gave him a standing ovation, Mr Zelensky thanked the European Union for its support with armaments.
"Without you, Ukraine is going to be alone," he said. "We have proven our strength."
"Prove you are with us, prove that you will not let us go."
Later the Parliament said it would look at a request by Ukraine for candidate status of the EU.
The Ukrainian president condemned the attacks on Kharkiv.
"This is the price of freedom," Mr Zelensky said. "This is terror against Ukraine. There were no military targets in the square - nor are they in those residential districts of Kharkiv which come under rocket artillery fire," he added.
Video footage showed a missile hitting the local government building and exploding, causing a massive fireball and blowing out windows of surrounding buildings. Freedom Square is the second largest city-centre square in Europe and a landmark of the city.
Residents of Kharkiv told the BBC they were living in "hell" because of apparent attacks on civilians.
Yulia, a married mother of three, said the Russians were targeting areas with no military infrastructure.
"They've been bombing... a residential area where tens of thousands of people live - mothers, children," she said. "We have very many injured... This is hell."
Another resident told the Newshour programme she and her husband and children were living in a bomb shelter because they did not know where the shelling would strike next.
Photo credit: Dmytro Zhyvytskyy BNO News Caption: The aftermath of the attack