by {"name"=>"Max Murphy", "avatar"=>"Murphy-CMU-logo.png", "bio"=>"PhD, Bioengineering", "location"=>"Pittsburgh, PA", "employer"=>"Carnegie Mellon University", "pubmed"=>"https://www.ncbi.nlm.nih.gov/pubmed/?term=murphy+maxwell+d", "googlescholar"=>"https://scholar.google.com/citations?user=CUGkYf8AAAAJ&hl=en", "email"=>nil, "researchgate"=>nil, "uri"=>nil, "bitbucket"=>nil, "codepen"=>nil, "dribbble"=>nil, "flickr"=>nil, "facebook"=>"Max.Murphy11", "foursquare"=>nil, "github"=>"m053m716", "google_plus"=>nil, "keybase"=>nil, "instagram"=>nil, "impactstory"=>nil, "lastfm"=>nil, "linkedin"=>"maxdmurphy", "orcid"=>"https://orcid.org/0000-0002-8559-2361", "pinterest"=>nil, "soundcloud"=>nil, "stackoverflow"=>nil, "steam"=>nil, "tumblr"=>nil, "twitter"=>"MaxwellDMurphy", "vine"=>nil, "weibo"=>nil, "xing"=>nil, "youtube"=>nil, "wikipedia"=>nil}
My thoughts on academic service.
I see service as the unexpected contributions I make to any team with which I identify. This atypical definition requires an explanation of team membership and what that means to me.
Contributing helpfully as a part of a team is fundamentally important to me. Whether it has been cheering for the Kansas City Chiefs, leading my teammates as co-captain of the Stanford fencing team, or working during the small hours of the night when my assistance is required on a project, I have always relished the feelings associated with surprising my team members through some unexpected contribution or effort. "Team player" is at the core of my identity.
Due to my existential fixation on team membership, I do not mind contributing to teams in ways that do not "earn credit." In basketball, a player exhibiting these qualities would be described as having a lot of "hustle," "intangibles," or sometimes "basketball IQ." By striving to exert the "intangibles" as they apply to all aspects of my own life, I hope that I am able to touch the lives of those around me in a positive way. Therefore, I refuse to embed a laundry-list of events, committees, or other politically-motivated tasks on my personal website. I believe stating this refusal outright says more about the kind of person I am (and at least gives a fair indication that I am a stubborn person). Please enquire with anyone whose team I've had the pleasure to be a part of, if you would like a fairer evaluation to this question than I could ever provide.
In this description, service is heterogeneous and almost always specific to circumstances (and the individuals involved). Team contributions counting as service are "unexpected" not because there is no expectation of a contribution; it is the contrary: individuals going beyond what is "reasonable" and producing a contribution that was not requested of them. In my experience, such an "unexpected" contribution has resulted many times from the incredible intuitions or efforts made by my collaborators causing me to pause, reflect, and appreciate my good fortune in having the opportunity to work with such incredible teammates. Therefore, my personal definition of service is a synthesis of my gratitude for all of my teammates and the associated self-identification I have formed with my cherished teams: I will continue to strive for excellence so that my own contributions to any team project could produce the same joy or inspiration in my teammates as they have produced in me.
Broadly, peer review is the act of appraising oneself of the work of peers within a field dedicated to the inquiry and advancement of knowledge pertaining to a specific subject matter. In this process of appraisal, the appraiser may:
The list could go on, but it is evident that there are many reasons why it is good that we read one another's works, try our best to understand them, and hold open discussions within our application-specific communities of practice. From this list, it can be seen that the act of peer review encompasses many benefits for both parties that engage in it. Without contrived barriers, there are really only three (or maybe only two-and-a-half) requirements for this process to flourish, conferring its many corrective and predictive benefits: time, media, and sensibility.
It is not without good reason that the popular notion of the absent-minded scientist emerged. The time required for completing a given task is often trivialized in academic science. In the context of peer review, time is simply the duration that an appraiser of a given work spends interpreting its contents. Methodological complexity of experiments is ever-increasing. Many disciplines require expertise not only within the life science domains, but also in math, statistics, and engineering as well. Therefore, the time investment required to simply read and truly understand a paper is non-trivial and must be acknowledged as such. Alarmingly, in my experience this has been glossed over or worse: many times when I go to discuss a paper it seems that I'm the only one who has read it. I'm convinced that this is not the fault of my peers, but instead it is a troubling outcome related to the hyper-incentivization on the generation of "papers." This adverse incentive schema is fundamentally coupled to traditional peer review. Unfortunately, the net result is a colossal waste of time and resources of exceptionally talented individuals.
Although the printing press was in fair use around Europe by 1500, it was not until over 150 years later that the first academic journal was published. It would not be until 1731 that the first fully peer-reviewed journal was launched by the Royal Society of Edinburgh. So we can see that from the outset, even the traditional peer review model was somewhat slow to gain traction. Clearly, science (even in its modern form) was able to exist and advance itself without a bureaucratic hierarchy of journal editors and review panels. Nonetheless, the advent of a standardized media (the journals) for aggregating and disseminating scientific evidence (and the associated methodological underpinnings related to the experiments producing said evidence) surely improved the rate at which new knowledge was acquired, curated, and logically sorted out.
While sensibility is not a "strict" criterion for the peer review process, my anecdotal experience in dealing with a reviewer who (in my opinion) asked a stupid and pointless question that was only there to delay the publication of the results that would ever be of interest to others was "just put up with it because it's better to not offend them." I don't see that sentiment as having any place in scientific inquiry, and perhaps that's why I'm writing this piece to begin with (it should be noted that I did address the reviewer's comment--it was unhelpful--and the manuscript was published). My opinion is that the whole business of "having peers" implies the necessity of this component. Vonnegut might have summed this up as "just be good, damnit."
I use traditional to qualify peer review with reference to the cumbersome process of disseminating results of a work as a manuscript. This occurs after scientific experiments and analysis of the results have concluded, or for creative works, once a "thing" has been invented/discovered and validated. Unfortunately, this process of dissemination has become erroneously conflated with the act of peer review. This confusion was seized upon (I didn't research the possibility of perpetration) by villainous publishing syndicates to spuriously inflate their importance and turn academic publication into a "sales" act. As a net result, arguments with peers in which the sanctity of journals is cast into doubt automatically prompts pompous questions of "so you're moving into industry?" as it is automatically assumed that you are somehow against the categorically good peer review.
Because of the intricately linked relation between peer review and scientific inquiry, and the heretofore marriage of peer review to journal publishing, it is not hard to see how this came about. Today, the traditional metric by which scientists are gauged is the ability to produce meaningful and impactful work (in the form of journal articles). Often, as a heuristic, the standing of those journals is used to imply the value of the scientist. Therefore, the whole process of publishing an article has become manufactured in the extreme. To simplify this charade, it could be thought of as a three-step process:
The first step is good: immediate interaction and feedback with peers who are immediately adjacent-to and interested-in the results of the reported works. This is often quite productive collaboration and involves direct feedback from experts in disparate fields whose blended insight and background serve to directly enhance the interpretation and reporting on experimental results.
The next step requires formatting things into a "final draft" that is acceptable to the senior authors on the paper. As the graduate student progresses through his or her career, it is most likely that in the beginning the senior authors will need to take an active roll and hopefully they do this in a way that is instructive to the student. My experience is that I've benefited from that interaction and I'm grateful for that structure.
While this relationship should benefit the graduate student in the beginning, it seems only fair that it should benefit the senior authors by the end. If the student is prudently developing his or her experiments that are based on furthering some particular knowledge frontier, he or she should quickly become a world expert in that hyper-specific application of knowledge. The experimental design, which in the beginning is most likely guided by the overarching themes and goals of the senior authors in the lab of the graduate student, likely addresses a question that is fundamentally important to one or more grants funding the lab. At this point, rather than an instructor, the senior author takes a less-active role in the development and direction of the paper. A primary motivator for the student to have the senior author on the paper is the "name-brand recognition" of the more prestigious senior author, coupled with the fact that the senior author's grants most likely pay the submission fees. These depend upon the journal, but will typically run you \$1k - \$2k depending on which options are selected and how much you want the editor to "like you" (looking at you, "Golden Open Access Fee"). To the senior author's benefit, the work is cited and "claimed" on the supporting grant's inevitable progress report. So we can see that this is "how the world goes around," when it comes to the business of academic publishing--at least before those pesky reviewers enter the picture.
By the time that the publication enters the "peer review" process, approval becomes a game of chance. Younger reviewers, who may feel that they have something to prove by being newly invited to this arena in which they have gained some ability to influence any decision-making, may provide unduly harsh criticisms. Conversely, senior reviewers may approve a manuscript that they really have no business evaluating.
My experience is that I have never been requested to assist in the peer review process (not entirely true, but of the requests that I've received they struck me more likely to be scams in the traditional sense than academic texts). I have spent an inordinate amount of time working with the rat pellet retrieval task, to the point that I believe I could be considered an expert in that area. I presented my work at the annual society for neuroscience meeting twice during the poster sessions. During that time, I investigators from around the world spoke with me and some of those investigators also study rat pellet retrievals. I also actively sought out investigators during their poster sessions in order to make connections. So, while I know which investigators I would request, no request has ever been made for my input on their experiments.
Unfortunately, the optimistic interpretation dictates my unsolicited opinion was due to the assumption that I do not possess the faculty to provide helpful insights or additions. The alternative is my opinion was actively avoided, for fear that I could produce a sound rationale as to why the study should be swiftly rejected. I think this second possibility is unlikely; I do not pretend to have an IQ of more than 100, and most of the time, I simply get by with hard work. It takes me a long time to get through journal articles but I read them to the best of my ability. Therefore, I could understand the former interpretation (although I think my opinion could be helpful nonetheless). I certainly am not looking to inhibit the ability of a hard-working scientist to share and disseminate the results of their difficult experiments.
Regardless, I think it is fair of me to express, at an emotional level (and in my own blog post), disappointment that my opinion was never sought by those whom I considered as my peers or better. Fortunately, as I will argue in the following opinion section, I've been able to seek out and actively engage with the scientific peer review process by simply identifying pre-prints and making comments on them or e-mailing the author directly.
At the current stage of my career (having just defended my doctoral thesis), the question of service in an academic sense seems to largely pertain to conducting peer review. It is probably not related to serving on committees, sitting on review panels, or in general doing things involving decision-making and resource allocation. I refuse to participate in journal-initiated peer review. This post clarifies my position, which does not dispute the importance of conducting peer review.
The act of publishing articles in journals other than Nature or comparable is unethical. Publication in other journals dilutes knowledge and fundamentally threatens the true institution of peer review. Worst of all, predatory publishers prey on the unknowing hubris of well-meaning scientists as a means to recruit their free labor.
To detail why this is the case, I will describe why:
It's unbelievable that I have to address this question as if by refusing to participate in a ritual I am somehow committing sacrilege. Let's start with the obvious: why do the publishers say that you, dear reader, should participate in peer review? Fortunately, I can pull that one directly from the source:
What benefits can researchers enjoy by working as a peer reviewer?
Being asked to peer review is a great honor. It shows editors consider you an authority in your field. You’ll keep abreast of research, learn new and best-practice methods, and start examining your own research from that critical vantage point. Peer reviewing helps you to become a better writer, and perhaps ultimately a more successful published author. All this work is evidence of your standing and contributions in your field, which can boost your CV and help you get ahead. Most importantly, peer review improves research. So if you are keen on helping push humankind forward through sound science, then peer reviewing is one of the most rewarding things you can do.
Just reading this quote from the Taylor and Francis website makes me sick, but let me break it down:
Being asked to peer review is a great honor.
Translation: we are pumping your ego, you great scientist and worthy person. By doing peer review, you can be good and honorable like us!
Translation: you are only a scientist if editors think you can do science well, not based on whether or not you can do science well.
You'll keep abreast of research, learn new and best-practice methods, and start examining your own research from that critical vantage point.
Translation: because we know that you would never ever read an article if you weren't required to in order to boost your own status.
Translation: okay we admit it- we really don't even care about your science, what's important is that you write papers.
All this work is evidence of your standing and contributions in your field, which can boost your CV and help you get ahead.
Translation: remember, science is not a team sport, it's about getting ahead of that other guy who might get your grant money!
Most importantly, peer review improves research.
Translation: but if you thought we were going to explain how that is the case, you are sadly mistaken...
So if you are keen on helping push humankind forward through sound science, then peer reviewing is one of the most rewarding things you can do.
Translation: remember, we're in charge around here and we just told you that this is "good" for "science."
Insofar as "academic teams," I see myself as a part of some larger and abstract entity chipping away at the wholly inadequate state of healthcare when it comes to rehabilitation from injury to the nervous system. One of the basic ways to contribute to that team is to provide feedback in the form of peer review. However, peer review has changed its meaning in the past 20 years. Today, team membership that requires peer review need not overlap with traditional peer review. More specifically, it is no longer necessary (other than for practical means of self-advancement) to go through the cumbersome peer review process that has been the province of academically-mediated journals for the past few centuries. This is because of technological advancements that have radically altered the communication media we use in peer review to begin with. Today, there are only two media requirements, and they are both inexpensive.
I don't believe there is any argument to be made regarding paper versus electronic formatting. It is frustrating that I am required to cater my paper format, which could be improved tremendously by including interactive figures (as would be done by any consulting firm making a dashboard for clients), to dinosaurs who "want their printed copy." It is even more infuriating when I am held up in the process of reporting results by feedback indicating that someone "does not like my drawing of a rat" or "thinks the color scheme is funky." I happen to hate traditional color schemes because my color blindness makes certain colors difficult for me to distinguish.
Modern peer review is the act of curating and responding to freely-available scientific content. It takes place during one's own spare time. Because most content is available through archival repositories available via the Internet, the media is digital. Constructive feedback is provided either in the form of site-specific comment threads or direct e-mail to the corresponding author. This results in active and immediate attribution of reviewer status to a peer, direct critique of a work to the authors, and removal of "editors" in the traditional sense.
Modern peer review is performed actively, frequently, and on an individual basis.
Modern peer review does not require an invitation. Up-and-coming scientists with good ideas can provide constructive feedback that would be just as helpful or valid as someone who has been in a field for 20 years. Similarly, someone who has been in a field for a long time is no longer able to stagnate a field simply because they do not agree with trivial semantics of some term or phrase.
Modern peer review is driven by a need to stay appraised of work that is directly related to subjects of interest.
Rather than serving as a strategic point of acknowledgment that is carefully and prudently developed via social networking that occurs over tens of years specifically for the purpose of augmenting the competitive appeal of requests for funding (i.e. grant proposals), the act of performing modern peer review is born of a desire to simply stay appraised of the current state of understanding within a scientific field. This means that modern peer review is a more sincere form of scientific inquiry and advancement than traditional peer review.
Modern peer review is efficient.
The fastest way to receive critical feedback about an idea or concept is to post it on the Internet. Most modern journal articles contain at least one author in the author list who has gained a following in his or her respective field. As long as a pre-print article is associated to the most recognizable author's Google Scholar account, then other responsible scientists will receive a notification that the pre-print has been uploaded and/or edited. This is a much faster way to notify the relevant parties of new works in their field than the traditional model, which relies on a third-party system (leading to the next point)
Modern peer review is not mediated by a third-party.
The journal-based model of traditional peer review requires an editor or editorial board. This entity is charged with fielding incoming works (which typically contain a cover letter describing the relevance and suitability of the works contained to be published in a given journal), making an initial determination of their suitability to send out for review, then sending the works to peers capable of their constructive evaluation. Selection of peers depends on a number of factors, but will involve some combination between the list of reviewers suggested by the manuscript author(s) and those the relevant editor deems acceptable. Unfortunately, by the time a journal editor becomes a journal editor, his or her closest peers may also be similarly busy and therefore unavailable to field such requests in a timely fashion. Furthermore, the more that the editorial duties increase, the less-engaged the editor becomes with the overseen area of scientific inquiry.
Modern peer review does not require an editor. Peers are directly connected either by outright exploration (on the part of the reviewer, who is seeking knowledge), or by past interactions (a Google Scholar citation alert, which provides an e-mail notification as soon as a followed author updates their Google Scholar profile with the newly submitted article). This latter connection is likely formed by peers with similar subject interests as they learn of one another's work via independent study and/or through meetings and shared discourse at conferences and workshops. None of these things requires an editor or fancy overhead fees.
Modern peer review is not politically motivated.
Most prestigious journals require a "novelty" component implying that included work is only suitable for publication if it addresses a particularly new idea, resolves a critical challenge, or raises substantially interesting new questions about the relevant subject matter. Work that validates or extends a past idea does not meet this definition. Therefore, groups with similar or competing objectives have an adverse incentive to "delay" the publication of scholarly works by hampering publication and dissemination in high profile journals by framing pointless inquiries in the format of seemingly constructive criticisms.
Modern peer review puts the burden of responsibility for gating public dissemination of research results (or their retraction) onto the individual researcher. If your work is bad or incorrect, take it down or run the risk of widespread public ire! The proof is in the pudding, so to speak. In order for this to work in practice, sites like arXiv (and its subsidiaries) must find a better way to permanently record the history of changes for any work that has been submitted. This way, any mistake persists forever in the public eye. It should be noted that this is not a challenging problem for markdown and html text-based documents: all that is required would be a system similar to that used by `git` for code repositories.
The journal-based model of traditional peer review had its place prior to the Internet. Two major objectives of journals was:
Provide a way to disseminate works to a large audience.
Provide a common contact point for experts within a field to give feedback on one another's works.
Journals, in their current formulation, are obsolete. The only value they render is "value recognition." They do this one job poorly (except you, Nature).
Too often, I hear these fundamental purposes ignored in discussions about the role of academic journals. Somehow, the focus becomes whether "peer review is dead." Peer review is not dead! While the journals may die, the scientific process (and on its coattails, the concept of peer review) will endure.
This is absolutely the strongest argument and explanator for the state of the current system. It does not need to be an argument. There are many tools for tracking individual contributions, which are championed both inside and outside of industry. We would do well to learn from software developers, who have been comfortable with version control for a long time due to the fluid nature of code development. Social hierarchies and bureaucracies will naturally reform as more-frequent direct interactions are enabled, allowing those who can contribute value to do so in whatever capacity they are able.
In summary, the traditional peer review model served an important role and is intricately tied to physical press. With the rise of decentralized, digital content, the journals have to seriously retool their arsenal and part of this will be coming to terms with exorbitant pricing and archaic philosophies about what peer review is. The traditional model, in my experience, is about elitism, connections, and nepotism (in the sense of an "academic family" tree). The rise of so-called "anti-science" movements in Western countries should be an important cultural litmus indicator that the academic norm is totally and completely unacceptable. Moving forward, the academic process needs not only greater transparency and data sharing, as have been repeatedly been called for in the last decade, but the complete and total abolition of the archaic journal-based peer review system. Pre-print archives, such as arXiv are on the right track, but need to make subtle but significant improvements before the system can make the "full jump."
As tenured investigators persist in already-saturated fields, it directly drives an increase in competition for scarce faculty positions. These coveted positions are rightfully claimed by those with a track record of excellent publications, which are more-often-than-not in journals that have high impact factors within their respective field. To be clear, despite many criticisms of its contents, I actually enjoy reading the journal Nature. The very high quality of work presented therein is exemplary and the exposure to subject matter across a wide range of fields that I would never think to explore otherwise is invaluable. As long as the very best journals maintain their rigorous standards of excellence, I believe there will always be a place (at least to some extent) for a few journal services in academic literature. We just can't all be Nature. I would just appreciate it if we could stop pretending that other journals even try, or that publishing with them somehow makes your work credible or good. It doesn't.
The first to go must be the worst perpetrators of egregious injustice: the "pay-to-play" model contrived by predatory publishers such as Elsevier. During my graduate studies at the University of Kansas, I often couldn't access literature because an article was behind a paywall. A library loan request could take weeks to arrive, and am I really going to do it if I am not even 100% sure that the article is relevant to begin with? After all, the fact that I am looking to read the article may mean I am uncertain of the methodological elevance of the work to my own to begin with.
A particularly hilarious example is that I could not retrieve the final (formatted) version of my own manuscript published with Journal of Neural Engineering. I can only imagine the profiteering owners of such publication syndicates twisting their cartoon-villain moustaches as they cackle:
"...and then we wrote in requirements that they would have to take ethics courses to learn about the "ethical perils" of "self-plagiariasm" bwa-ha-ha-ha!"Please, these heinous villains must be stopped.
None of this works without you! The following steps describe ways you can help with modern peer review either as an author or as a reviewer.
Remember to exercise sensibility when you give feedback on others' work: modern peer review is not an excuse to anonymously blast another's hard work, no matter how strongly you may feel about its contents.
Until the full team of scientists realize how badly they're getting ripped off, things will not change. However, I firmly believe that we can change, and surely we can do it faster than the last revolution in peer review brought about by the printing press. If we want Trumpers to stop yelling at scientists, we can start by truly taking the difficult first steps required by this exercise in humility.
tags: service - academia - peer review