More than 80% of people with facial disfigurements have been abused or harassed in the street, according to a new report.
A small survey for the charity Changing Faces also found half of those with a disfigurement feel vulnerable on public transport.
The report looked at how people's lives are affected on a day-to-day basis.
Newsbeat's been talking to Louise Quarmby who was born with a cleft lip and palate.
"The first 20 years of my life were basically spent going in and out of hospital," says Louise.
Louise's condition is caused when parts of a baby's face don't join together properly in the womb. It means there's a gap in the roof of the mouth or lip.
It's the most common birth defect in the UK.
Despite that, Louise has always been treated differently.
"Since I can remember, people have pointed at me, stared, made comments behind my back," says the 28-year-old from London.
"The first time I can remember was when I was about five. One of the other kids said: 'Did someone push your nose down and then the wind changed?'
"Even adults would point it out.
"You never get used to it. It's a constant barrage every day - of stares, double takes."
Louise's experiences are backed up by the Disfigurement in the UK report. It highlights that in an image-obsessed society, it can be very hard if you look even a bit different.
"You have to mentally prepare yourself every day. Who's going to stare at me, who's going to make a comment?"
Louise has even had people take photos of her while on public transport.
"The worst is when someone makes a comment about me and no-one else who's around does anything to support me. They just sit there in silence. It's so embarrassing."
But Louise understands why she gets the looks and the comments.
"If you haven't seen something before, it's normal to fear it. The more people see something, the more normal it becomes. That's why we have to educate people."
Louise is pleased she was a teenager just before Facebook became massive.
"My teens were the worst. I had very low self image and I don't think all those photos everywhere would've helped. I didn't know anyone else who looked like me.
"There are so many horrible memes with people teasing others who have physical conditions."
Newsbeat's recent documentary on memes looked at what happens when things get nasty on social.
Craig Byrne told us how we was mocked when his face was turned into a meme.
"I just wanted to go to the person and say, 'Why are you doing that - why would you put that up?'"
But Louise says social media has been good too - hearing positive stories from others with similar conditions.
Now a TV producer, she says she's in a much better place then when she was a teenager.
"I vowed after my last operation that I'd be happy with my appearance. The problem's not with me, it's with them."