Dr. Michelle Wong, chemistry Ph.D., science educator, and cosmetic chemist explained the flaws in the aforementioned rosemary-helps-hair-growth study that many social media influencers cited as evidence in a recent video. She also explained in more detail how peer-reviewed studies work and how we as consumers (of media and products) can spot bad science.
Wong points out some problems with peer-reviewed studies in one of her videos. Not all topics get much attention in the science world. Papers published in fields such as epidemiology, genetics, climate change, or artificial intelligence get the most attention nowadays. Naturally, there's less scrutiny for less popular topics. Not a lot of scientists are discussing and critiquing the studies in less popular fields; thus, that research can be less reliable.
The other problem is that peer reviewing is voluntary, unrewarding work. Scientists don't get paid for reviewing (especially reviewing well) new research. It's anonymous and variable, Wong points out, so the quality can vary. Sometimes peer reviewers might not read the paper as closely if they know it might not get a lot of attention in the future.
Lastly, peer reviewers aren't safe from personal bias. When a paper has a well-known author or one that comes from a prestigious institution, peer reviewers might not give it much scrutiny. "It's actually a really well-known hack for getting papers published quickly with minimal revisions – just stick a well-known author on there," Wong explains.
Michelle also gives tips on how to spot bad science (never trust an abstract!), so be sure to check out her video if you're interested!
