A second woman was found dead inside an apartment building on the same street.
"All the information we have at hand would link these two crimes," police commissioner Mick Fuller said at a press conference.
He said the suspect was a "lone actor" with a history of mental health issues.
He was known to police but "his history was unremarkable compared to the gravity of the crimes he committed", Mr Fuller said.
The police commissioner said the suspect had some ideologies related to terrorism, but had no apparent links to other terrorist organisations.
It is believed the man had a USB drive containing details of mass killings in New Zealand and the United States.
Mr Fuller said the stabbing is not currently being classed as a terrorist incident but this would be reassessed as the investigation continues.
Three "heroes" were praised by New South Wales Police Minister David Elliot for their role in detaining the man and acting "well and truly beyond the call of any citizen". He also congratulated the firefighters involved.