Modern professional matches are routinely punctuated for rulings by the TMO on everything from the scoring of tries to foul play, offside in the build-up to a try and off the ball incidents.
Owens said he accepts the reasoning but felt the option was being over-used.
"I can understand that people at games want to see the same thing people do at home, or in the hospitality boxes, but replays can make a huge difference -- in players reacting on the field, supporters reacting," he said.
"I think the TMO is used too often and for too many things," said the Welshman, who refereed last year's World Cup final.
"Personally I think we should go back to where it was five six years ago where it was only on the goal line -- 'try yes or no?'
"Then, as a team of officials, we would have to work harder and be better at making the decisions that don't involve the goal line. Things like obstruction, did he come in from the side, forward pass? There are so many things that can go either way and I think you'd probably find that if you make the decision and it's not replayed then nobody argues with it.
"I also think that would help reduce the issue of players questioning the decision as most of the time they won't have seen it at all."
Owens was taking part in a debate at the World Rugby Conference on whether respect for referees in the sport was in decline.
Rugby is immensely proud of the relationship between its referees, players and coaches, from grass roots to internationals, and other sports regularly point to it as a beacon of respect.
While Owens feels there has been some decline of late he does not see it as a major issue, though he is concerned that the TMO can often have an unwelcome influence at all levels.
"Boundaries are being pushed a bit at the top, others see it and follow that example, but I think that is probably a reflection of society," he said.
"I'm not sure if the TMO is eroding the referee's authority but it is possibly eroding people's perception of it.
"Sometimes a decision you make in the flow of the game might look a bit different in a replay, especially in slow motion, and you are forced to make a technical decision that maybe you wouldn't have made.
"Empathy is essential to good refereeing. You understand why something has happened but you might have decided it didn't impact on what was happening, but then it's on the big screen in isolation, the crowd's seen it, you've seen it, you can't ignore it and your hands are tied somewhat.
"You have to remember that rugby's rules are complex and there are a lot of grey areas. Two different refs might view the same incident differently but that doesn't necessarily mean one of them is wrong."
Owens has taken charge of almost 80 tests, more than any other referee, and due to his humour on the pitch and his high-profile off it, where he presents a TV programme in Wales and has spoken movingly about his decision to come out as gay, he is a hugely influential character in the sport.
Despite being in constant demand for the professional game he said that he still occasionally takes the whistle in community rugby, where he had his eyes opened to the changing atmosphere on the pitch.
He said he took charge of an under-13 game in the Welsh valleys recently when a promising 17-year-old referee came to assist as touch judge.
With 10 minutes remaining Owens and the teenager swapped places and he said he was amazed at the almost instant transformation.
"When I was reffing the behaviour was perfect, nobody questioned a thing, as you would probably expect," he said.
"But once we swapped it was like a different game -- they were arguing, gesturing, questioning -- yet it was the same kids.
"So we need to keep educating players that it's not acceptable at any level.
"We're never going to go back to where we were 20 years ago when everyone called the ref 'sir' but we do need to retain respect.
"I need to stress that I don't think we are in a bad place, but we need to be vigilant in making sure we don't get to one."