My first PLoS comment: High rating of an article on TSLP being the cytokine link between eczema and asthma

5/27/2009 SJK Note: After I wrote this, Bora Zivkovic sent me links to the PLoS community blog where he talks about commenting and rating PLoS articles. Both are very much worth reading! Bora is the Online Discussion Expert for PLoS.

Recently, William Gunn Mr. Gunn composed an excellent article discussing online identity and the making of public comments in scientific circles. Without immediately spiraling into a stream of ridiculous conversation, I can't really comment on his post, or the ensuing friendfeed thread. Suffice to say that Mr. Gunn and others on friendfeed inspired me to be a lot bolder in commenting on PLoS articles.

So, tonight I made my first comment on a PLoS article. Previously, I had viewed commenting on the actual article site as a very formal procedure that required attaining the highest level of understanding of the article before submitting a comment. Essentially, I was viewing commenting on an online article the same way I viewed submitting an official comment to an article published in Science or Nature (or other journals). Published comments in those journals are almost always refutations of the article that seemingly without fail lead to concomitantly published rebuttals by the original article authors. Thus, the culture of commenting on articles is fraught with nastiness and putting one's scientific reputation on the line. This could be the reason that so far "official" online commenting on peer-reviewed articles has been very limited, whereas "unofficial" or off-site commenting has been more common. By "unofficial," I am loosely referring to comments made anywhere that is at least one link removed from the actual published article site. For example, an external blog, friendfeed discussion, or notes left on article managing services such as citeulike.

It occurred to me while laughing and crying my way through the recent friendfeed discussions (OK, fine, here's a link to perpetuate the madness) that this culture may be relatively easy to change. (Aside from any questions of whether it's necessary to change.) In my opinion, PLoS has already made one innovation that vastly increases the odds of a user making a public "comment." They have separated the article ratings into three categories: Insight, Reliability, and Style. From my personal experience, that opens the door almost all the way in terms of inviting some kind of reader feedback. Rating an article on "Style" does not carry much professional risk from my viewpoint. Rating on "Insight" requires understanding of the possible impact of the article, and is thus much more weighty than the "Style" rating. However, I personally feel I can rate an article on "Insight" without assessing the quality or reliability of the methods and data. I recently did this with a PLoS ONE article I saw on single-cell sequencing of uncultured organisms. To rate an article on "Reliability," I feel requires the kind of in-depth understanding that would be required for me to send a formal letter into the editor of Science or Nature that could be published. Thus, the barrier for me to rate on "Reliability" is quite high. Especially since if I'm going to put in enough effort to feel completely justified in rating, it's likely to be less than a 5-star rating. (I guess I'm feeling like I spend more time reading articles that I disbelieve than those I do believe?)

Another reason that placing online comments does not have to be as formal and negative as with traditional published comments is that the comments are published without a delay waiting for the original authors to compose a response. This then reduces the expectation that the publishing authors must respond and therefore takes the formality down a bunch of notches in my opinion. Also, in terms of PLoS the whole mission of the journal is to make research more broadly and rapidly available--and thus I think there is an expectation that the comments should also come from a broader base of readers.

So, that is what inspired me to take the time to read a PLoS Biology article and compose my first online comment tonight. I was also inspired by the belief that we're still very early in the process of dictating the culture of online discussions of peer-reviewed research--and thus a concerted effort can make impact in what ends up happening. This inspiration was combined with the coincidence that my wife sent me an article from BabyCenter today that caught my interest because it was discussing the recent PLoS Biology article. Finally, the thing that finally tipped the balance and convinced me to take the leap and make my first PLoS comment was a healthy dose of "WTF" So I stopped worrying and took the leap. :)