The Game of Peer-Review
As the saying goes: When the ancients abolished Gladiatorial games, they introduced Academia.
Recent years have exposed those we are expected to refer to as our de factor authority on any endeavor of scientific import. The so-called ‘experts.’
‘Expert’ is a construct. It refers to a person with a set of characteristics.
The Expert is:
well-educated, preferably from the Ivy League
eloquent, or at least, well-rehearsed
well published, and reviewed by his peers
accepted by mainstream institutions
conveniently comes to conclusions that align with State objectives
In some cases, the Expert is also the academic that is willing to pander to an audience. Tell them what they want to hear. Tell them what they think they need to hear. Say things which move them emotionally.
In the most generous of interpretations, the Expert can also be the one who is able to convey complex ideas in a simple and understandable way to the general public.
In the realm of academic medicine, an important marker of your ‘expert’ status is the length of your CV. Specifically, the section of the CV concerning peer-reviewed publications.
As I have mentioned in prior articles, “peer-review” is the currency of academia. It is the denomination by which students are chosen for education, graduates are chosen for training, trainees chosen for faculty positions, and all of the above chosen for grant funding.
Unfortunately for the unaware public, once you incentivize these metrics…it becomes a game.
Let the Games Begin
The publication game goes beyond hiring practices.
Peer-review is an important metric by which faculty are chosen for promotion, as well as new opportunities within the academy. By virtue of its relationship with grant-funding, it is also an avenue by which professors can hope to conduct research of their own authentic interest.
It also determines opportunities for collaboration with colleagues, as well as interest from students who may want to work with you.
Tit for Tat
If Professor X has a reputation for getting published, then he will be sought after by Professor Y. So, when X publishes…Y also adds a line of publication to his CV.
However, Professor X may only choose to collaborate with Professor Y if he believes that Y will also publish work of his own…and add Professor X on the list of co-authors.
Let me clarify what I am insinuating here.
In academia, people will add the names of their colleagues as co-authors even if they have not materially contributed to the work. This is fraud, and it is rampant.
It is also a deal with the devil, so to speak. Because, if you accept the offer of being added as a co-author, you will now be expected to return the favor.
If you don’t return the favor…there will be consequences.
For starters, Professor X will no longer collaborate with you. They may also start spreading word that you are not a “team player.” Not one to “keep the ball rolling.” Difficult to work with, and so on.
Stacking the Deck
I have already written about the role of peer-review as a status-quo maintaining tool of academic censorship.
Before hanging up my academic cloth, I had the privilege of working with some of the most renowned physicians in my specialty.
Through this journey I was not prepared for what I had encountered in the field of hyper-specialized publications.
Publishing derivative drivel and tit-for-tat games aside, I encountered a rather interesting phenomenon.
When you submit a manuscript to be considered for publication, there are several steps. One of these steps involves the assignment of peer-reviewers who critique your work, and help to prepare it for publication to the standards of the journal.
Once the work of a researcher gets specialized enough, there remains only a handful of peers to review your work before it can get published. More often than not, these peers are colleagues, coworkers, or past collaborators. Most editors do not know who these people are. They have to seek them out. This can be quite difficult and time-consuming.
This is an obvious problem that requires a solution. Not only is this a problem for generic high-impact journals such as Nature, but also for specialized medical journals. One of the solutions these publications have implemented is the option to suggest reviewers.
You may think that this is a rather benign and necessary process of publishing highly specialized research. From what I have seen, there is far more than meets the eye.
For starters, this process opens the door for a natural extension of the tit-for-tat game described above. When academics want to reduce the timeline and obstacles to publication (because they are in a race…as they frequently are), they may “suggest reviewers” with whom they have worked with on other collaborations.
When the journal reaches out to the suggested reviewer, this colleague is yet again faced with a moral dilemma.
Do I perform my duty as an honest scientist, or help my colleague in the progression of their career? After all, one day I may also need their help. I trust their work, right? I’m sure it’s fine. Let’s approve it.
Once these reviewers who ‘keep the ball rolling’ get enough manuscripts under their belt, the journal starts to view them as reliable and invites them to take a position on the editorial board.
By virtue of these of reviewers obtaining positions of authority within major medical journals, the corrupting practice of ‘peer-review’ rapidly gains foothold within the industry.
Whereas in the past you had a bottom-up corrupting force initiated by a game of bad incentives…there emerges a top-down enforcement of these practices which only amplify the bad ‘science’ that promotes the Experts.
Where does all this lead?
Well…that’s a question we are all struggling with. It is why this Substack exists.
It is because of mechanisms like these that the future of licensed medicine seems dismal to an increasing proportion of physicians in the West.
Twitter actively throttles exposure of these Substack articles to my social media followers. I would greatly appreciate if you take a moment to like or retweet to increase visibility.
Verified. I've been in and around academic research for most of my life, and I've seen the problem growing and metastasizing since 1970. It's been discussed and bemoaned inside academia for all of that time, but never improved or changed.
The odd part is that outside writers don't seem to grasp the seriousness of the problem, even though most outside writers have been in academia at one point or another. I doubt they're naive; more likely they also want to keep it going.
It has been over a year since you wrote this, Doc. But very little has changed on the ground.
https://jessicar.substack.com/p/the-peer-review-process-needs-a-fix