Of brother Prince William and his wife Catherine, he said: "I always hoped that the four of us would get on.
"But very quickly it became Meghan versus Kate.
"And that, when it plays out so publicly, you can't hide from that," he told Tom Bradby, in an ITV interview.
BBC reports in the 95-minute interview to promote his memoir, titled Spare, he recalled the days when he was the "third wheel" at official engagements and other outings - but his relationship with Prince William and Catherine was particularly warm at that time.