Kane Tanaka, 118, was supposed to take part in the relay for Tokyo 2020 in Fukuoka, southern Japan, on 11 May.
The relay, which began in March, has been hit by a coronavirus outbreak, with eight cases linked to the event.
Last month, amid a surge in cases, Japan declared a state of emergency in major cities including Tokyo and Osaka.
Several celebrities have also pulled out of the torch relay over concerns about the virus.
Ms Tanaka, who was born in 1903, was recognised by Guinness World Record as the world's oldest living person in March 2019.
An official at her nursing home said her family had emailed the relay organisers, saying she "wanted to withdraw from the relay as she and her family were concerned about spreading the virus at the nursing home