Sam Altman, the CEO of OpenAI, the company behind ChatGPT, testified before a US Senate committee on Tuesday about the possibilities - and pitfalls - of the new technology.
BBC reports in a matter of months, several AI models have entered the market.
Mr Altman said a new agency should be formed to license AI companies.
ChatGPT and other similar programmes can create incredibly human-like answers to questions - but can also be wildly inaccurate.
Mr Altman, 38, has become a spokesman of sorts for the burgeoning industry. He has not shied away from addressing the ethical questions that AI raises, and has pushed for more regulation.
He said that AI could be as a big as "the printing press" but acknowledged its potential dangers.
He also admitted the impact that AI could have on the economy, including the likelihood that AI technology could replace some jobs, leading to layoffs in certain fields.
"There will be an impact on jobs. We try to be very clear about that," he said.
However, some senators argued new laws were needed to make it easier for people to sue OpenAI.
Mr Altman told legislators he was worried about the potential impact on democracy, and how AI could be used to send targeted misinformation during elections.
He gave several suggestions for how a new agency in the US could regulate the industry - including giving out and taking away permits for AI companies.
He also said firms like OpenAI should be independently audited.
Republican Senator Josh Hawley said the technology could be revolutionary, but also compared the new tech to the invention of the "atomic bomb".
Democrat Senator Richard Blumenthal observed that an AI-dominated future "is not necessarily the future that we want".
"We need to maximize the good over the bad. Congress has a choice now. We had the same choice when we faced social media. We failed to seize that moment," he warned.
What was clear from the testimony is that there is bi-partisan support for a new body to regulate the industry.
However, the technology is moving so fast that legislators also wondered whether such an agency would be capable of keeping up.