A couple of game-defining calls have gone against the Warriors this season - including the sin binning of Jazz Tevaga against the Eels in the last round.
The NRL's head of football, Graham Annesley, has since said Tevaga's sending off was wrong, but Payten doesn't know how to turn the tide in his side's favour.
"I just have the feeling that because it's us and the Warriors its forgotten about pretty quickly," Payten said.
"I don't know how that can change, do I come out and make a bigger deal of it post-match, I just don't know which way to attack that but I do get the feeling because it's us it's just pushed aside pretty quickly and we move on."
Payten believed other sides, particularly those from Sydney, fared better when it came to calls going their way.
Payten, who will coach the Cowboys next season, wants the NRL to take more accountability for referees who make the wrong calls in the big moments.
He thinks the referees performances need to be scrutinised and he has suggested the NRL use the same method he uses to select his players to choose the referees.
"I know that if it's one of our players and they continually make mistakes, or mistakes in crucial parts of the game, under pressure and it's a consistent thing where it happens week upon week, we don't pick him. I think that's the accountability that they should hold from their end, so that's the way I would go."