Open Access Week event at U. Arizona: Reproducibility, Open Data

Earlier this week I was lucky to participate in the Open Access Week event at the University of Arizona: The Future of Data: Open Access and Reproducibility.  The event was hosted by Chris Kollen and Dan Lee of Arizona University Libraries.  I am very grateful for the invite and the opportunity to meet them, some active member of the audience, and the other speakers, Victoria Stodden and Eric Kansa.

Victoria Stodden gave an excellent talk, framed around the computational sciences, and with the major point: Instead of promoting "open data," we should promote "reproducibility" in science.  She argued, very convincingly, that good science requires reproducibility and thus scientists should be easily convinced that we need very high standards for reproducible results.  For computational research, the only way to ensure reproducibility is to publish much more open data and open code than is normally done now.  If your result is computational, how can anyone hope to replicate and build upon your results if you haven't provided the source code and the data sets?  They can't, but publications without code and data are by far the most common these days.  It's a failure of science that is probably caused by many factors.  One that comes to mind is that computational scientists have been forced to fit their "publications" into standard peer-reviewed articles, where the system is not set up to accept and / or host source code and data.  (As an aside, this is clearly a routine failure of peer review, as referees obviously are not ensuring reproducibility of the research, which should be a primary criterion for publication.)  Scientists understand that reproducibility is an essential element of research.  For example, two years in a row, my undergraduate physics majors identified reproducibility as the most important element of good science (see brainstorming 2010).  Since scientists understand this, then they will naturally practice open publishing of data, code, methods when they realize that reproducibility is missing without those elements.  As Victoria argued, demanding "open data" leads to confusion and resistance and ultimately probably lack of compliance.  In contrast, demanding "reproducible research" is already a cultural norm and it naturally leads to open data and open code of the most helpful variety for reproducibility.  Victoria's slides can be found here.

The notes for my presentation can be found on linked mindmaps, starting here.  (Click on the tiny right arrows to navigate.)  My notes are probably not too meaningful if you weren't at the symposium.  In contrast to Victoria's high-level talk about policies that could make a major impact, I told a few stories about open data and open notebook science in our own teaching and research labs, and the successful impact we've had already.  I think (hope) it provided concrete example of the benefits of open science.  On the one hand, I showed that open science, especially open notebook science strongly promotes reproducibility.  This has been seen best in the undergraduate physics lab that I teach.  Students read the notebooks of other students from prior weeks and prior years.  They build upon these previous results, which allows them to get the experiment working much quicker and have more time to explore new aspects of the experiment, or to develop new data analysis methods.  They are doing real science!  I showed an example of an excellent primary notebook from Alex Andrego and Anastasia Ierides.  However, I think I also showed that open data and open science make an impact beyond just reproducibility.  This impact is in reuse and repurpose of data. I told two stories where theory and research groups already have been able to use data we publicly shared on youtube.  One group has already used our data in a theory preprint on the arXiv.  Both groups expressed delight and gratitude that our data was freely availalbe.  There are two important features of these stories.  First, both groups used our data for a purpose that we had not (and probably would not have) imagined!  Clearly the impact of our data was multiplied by being public.  Secondly, we did the easiest and simplest sharing method we could find: youtube, yet we still made an impact.  We are currently working with Rob Olendorf, a data curation librarian at UNM to vastly improve our sharing.  This will include permanent citation links, vastly improved metadata (at least 10x more than the data itself), hosting by the institutional repository (much safer than our lab server), and links to other data sets.  Reason would have it that if we could make an impact with the imperfect system we tried first, then the impact will be much higher with the data shared via Rob and the institutional repository.

The final talk was by Eric Kansa, who described the amazing work of him and his colleagues on Open Context, a platform for sharing and linking archaeological data.  His notes from the event can be found here.  And his slides are available also: A More Open Future for the Past.  Despite being far from the field of archaeology, it was easy for me to see the vast impact that Eric and his colleagues are making via the open context project.  A large amount of time, sweat, and money are expended collecting archaeological data.  Without opening these data and curating and linking these data, the potential impact is severely limited.  The Open Context team has developed a method for collecting these data, archiving them, and linking them to other data sets.  The method is very effective, and importantly requires far less work than required to collect the data in the first place.  This seemed clearly, to me, a case of the huge power of data reuse and repurpose. In contrast to computational science, the power of data reuse seemed to trump the need for open data for reproducibility.  This is not surprising, given how different the two fields are.  But it was an interesting and somewhat confusing contrast for me between the needs for open data in computational research versus archaeology.

There were several engaged audience members.  One of them was Nirav Merchant, with the iPlant Collaborative.  Victoria and I were highly impressed by the computational platform that iPlant has developed already, only three years into the NSF cyberinfrastructure project.  I was simply amazed and I couldn't do it justice describing it.  The ability to ensure reproducibility of computational research with the iPlant platform is vast.  One example is how easy it is to save an image of a virtual machine and then share this image with other users.  They demonstrated this for us and it took only a few clicks and less than a minute.  I highly recommend reading more about iPlant at their site linked above.  The iPlant team that we met was energized, engaged, and collectively brilliant.  I'd love to know how they assembled their team as they've clearly done an excellent job.  I intend to keep in contact with the iPlant folks and am even hoping that I could introduce the computational platform to my Junior Lab students this year.  I think the exposure to these state of the art and "open" tools will be invaluable for their future research.

Overall, the one-day Open Access Week event was highly successful for me.  I met some amazing people and gained a lot of clarity in my thinking about the imperative for much more openness and sharing in science. Incidentally, maybe not coincidentally, during my flights I was able to read Michael Nielsen's fantastic new book on the untapped potential of connected, open science: Reinventing Discovery.  Despite having met Michael and having heard him speak a few times, I still found the book riveting and I learned a lot.  I absolutely recommend the book to anyone interested in the practice of science!

ScienceOnline2010 -- Top N things I learned and was inspired to do at #scio10

• FriendFeed thread for this post
  

I had a blast at ScienceOnline2010 last weekend!  Thank you Anton, Bora and others who spent so much energy organizing it!  Approximately 250 people attended and it was a very diverse crowd of scientists, science writers, publishers, librarians, science outreach specialists, high school teachers, even high school students.  Much has already been blogged about the conference, including many "Top N" lists.  You can find a list of them here.  My favorite so far is Jonathan Eisen's "Enough w/ the good: here are the top10 problems w/ the #scio10 meeting."  Hilarious!  Despite everything having already been done, I hereby present my belated list of things I learned, things I did, and things I've been inspired to do:

1.  I had a ton of fun interacting with a bunch of e-friends, old and new.
I attended as a scientist with an interest in all of the other areas.  It was definitely a new feeling to be in a session and have the speaker ask, "how many of you are scientists?" with the answer being a very small fraction of the participants in the room.  It was also a very new and thrilling experience to finally meet in person many people I've known only virtually for the past year.  I think Cameron Neylon was the only person I'd met in person previously.  Despite that fact, it was incredibly easy to have conversations during lunch, between sessions, and of course at the bar.  I uniformly enjoyed these people even more in person.  An incomplete list of the people I had the pleasure of chatting with include: Bill Hooker, Jean-Claude Bradley, Pawel Szczesny, Walter Jessen,  Christina Pikas, Hope Lehman, Peter Binfield, ...  Plus I found some completely new friends at the conference, including Antony Williams, Greta Munger, Dorothea Salo, Andy Farke, Natalie Villalobos, Michael Habib, Annie Crawley, ...  Now, those previous two sentences could be perceived as egregious name dropping, which I am guilty of, simply because it's quite an amazing list of people, none of whom I knew before I became active in open science.  Clicking through those names and reading what they're saying, and you'll realize why I feel so lucky to have met them!
(Note, I forgot: Fabiana Kubke, ...)

2.  Antony Williams and I made some kind of running challenge.
I knew of Antony Williams from his ChemSpider fame.  I also had recently read one of his personal blog entries, about running 1000 miles in a year, along with his ugly and unfortunate calf injury.  I'm not sure I knew these were the same person, though.  Nevertheless, he walked passed me in the hotel bar, and I accosted him to inquire about his calf injury.  I'm pretty sure he didn't know me at all, but luckily we had on slick name badges and I was surrounded by credible people.  It'd be a big challenge for me to recount the conversation (I swear I remember it perfectly, I just don't feel like writing it down).  Let's just say that I was happy to learn that he'll be back running again within a week or so and that he has a goal of raising money to fight asthma.  I have had an idea mulling in my head that I could raise money and get motivated to get better at running by setting a race time goal.  I thought this was a perfect match with Antony's goal, so I quickly challenged him to a running competition.  He quickly agreed (fearlessly) and I tweeted/friendfeeded it to lock in the deal.  Over the next week, I'll see if I can clarify the challenge and I'll post updates to that thread.  I'm thinking I'll setup a Google spreadsheet for me and others to place their pledges and monitor the progress.  Antony already knows about Nike+ technology, and I'm looking forward to doing something like that too.  It'd be a good way to try out new things in open data, and open notebook science, actually.  Suffice to say that I'm going to get better at running, lose a lot of weight, and hopefully we'll raise some money too!

3.  I and KochLab are going to get better at doing Open Notebook Science and sharing data & software this year
I learned a lot at the conference about tools that exist for carrying out open notebook science and sharing data, methods, software, etc.  I still have a lot to learn, and indeed many tools still need to be developed.  But I know that our lab can make improvements this year.  Here's some concrete things that we'll do:

• I've sent an email to Amy Jackson, Digital Initiatives Librarian at UNM, requesting a meeting.  I had briefly spoken to her via email in October and now I'm fully energized to have a meeting with her and see what kinds of first steps we can take towards building a partnership between the library and our lab in terms of sharing data and conducting open science.  I'll try to leave updates on this FriendFeed thread.
• Get better at sharing software.  Currently we use LabVIEW, which is a wonderful programming environment.  One of the major benefits is that it's a graphical, data flow language.  This makes the code a 2-dimensional diagram...so in my opinion, it's exceptionally easy to read other people's code.  Unfortunately, it's a proprietary and expensive coding environment.  You are allowed to compile .exe and .dll files for others to use freely.  But that's not open source.  So, there are two routes to go: (1) We could compile virtual machines (VM) and send those to people for exploring our code.  For example, referees of papers we submit.  This was an idea from Deepak Singh at the meeting. Licensing is an issue here, and what I'd like to do is find someone at National Instruments (creators of LabVIEW) and discuss what can be done to serve our open source needs.  (2)  Learn a text-based, freely available language.  I think this would be valuable for our students anyway, in terms of building their resumes.  While at the meeting, I thought Ruby was a good idea, but now not so sure.  I've posted a FriendFeed message about this, and have received all kinds of very valuable advice.
• Adopt techniques to make it easier to capture our workflow in the lab.  OpenWetWare has innovations coming up soon and we'll certainly jump on those.  Cameron also hinted at something revolutionary coming up this year, but said he'd have to kill Bill if he told him what it is.  It wasn't clear that he'd have had to have killed others in the room, so I was disappointed he didn't say what it was.  (Just kidding, Bill!)   But the fact is, I don't think our current tools are nearly sufficient.  I'll put in more effort to make positive steps here, but I'm not sure exactly what yet.

4.  Improve the state of publishing, one article at a time: Try out some ideas via PLoS ONE.
I was delighted to have the opportunity to talk with Peter Binfield at the bar and discuss publishing ideas with him.  I have a lot of ideas that I'd like to try out, and it hasn't yet been proven that all of them are lousy.  I ran many of them past Peter and his general response was, PLoS ONE would love that -- it's just that whatever scientific community you're in may not respect it.  Very true.  However, I do think many in my community would respect anything that showed how publishing could be better by being different.  And I now realize that PLoS ONE is a very good platform for trying out a few things.

• At lunch with Bill and Pawel, they mentioned an idea that others had been talking about.  Unfortunately, I don't know to whom to attribute the idea. (Note: Bill and Pawel give attribution to Fabiana Kubke, which sounds right to me now.)  The idea is that publishing one very good figure would be a good idea.  I like this idea enough to think about trying it out.  It is related to an idea Larry Herskowitz and I were discussing a few days before the conference:  Can we publish a paper without an introduction?  Or an introduction that just says, "see such-and-such other paper for introduction?"  So much time is wasted rewriting introductions (in my opinion), especially when trying to avoid copying prior written work.  Is this necessary?  Publishing a single figure takes this even further.  I have some interesting data from grad school and postdoc that I have not been able to publish.  It pains me that it's just sitting around, useless.  Do you think it's worth using this data to test out the "publish one figure" method?   If I make progress on this, I'll post updates on this FriendFeed thread.  (Note added: Cameron Neylon reminded me that BMC Research Notes may be a better venue than PLoS ONE for this.)
  
• For maybe ten years now, I've thought that the value of anonymous peer review is overstated.  During that time, I've heard other people, much more eloquent than me also express this opinion.  Just briefly, I think fully-attributed, non-anonymous peer review would solve many problems that exist with today's science, and I discussed this a bit at the meeting.  Two of these problems are: (a) good referee work is difficult, and good referees are not credited for the work and the original ideas, and (b) a whole lot of incomplete and sloppy work is submitted and much of it is published due to ineffective referees / editors.  The solution I like is for every aspect of the peer-review process to be published, including the original manuscript submitted, all subsequent revisions, and all communication between authors, editor, and referees.  Clearly this solves problem (a).  As for problem (b), people often say, "but people aren't going to say negative things if their name is on it!"  First of all, that's not necessarily true.  Secondly, referees have the option to decline without comment.  If the editor cannot find someone willing to slam the paper, then it's just returned to the authors.  Voila!  One less crappy paper published.  The arguments get more complicated, especially when considering that grant review is anonymous, providing a lot of opportunity for underhanded retaliation.  I can't mathematically prove that it's a good or bad idea, so I'd like to try it out and see what happens.  It occurs to me that I could submit a manuscript to PLoS ONE and request that the Academic Editor implement this idea.  Why not?  Should be OK as far as I understand the rules.  I don't expect to submit garbage, so it won't be a complete experiment.  But something may be learned, and at least all the referees will get credit.  I may update progress on this idea on this FriendFeed thread.

OK, That's enough for now.  I can always add more later, especially since I aptly named this post.  For example, I may talk about the rebranding of our blogs that Walter Jessen recommended.  It's a good and valid suggestion, but I'm not adding it yet, because I'm not sure how soon I'll be able to think about that :)

A PLoS ONE Success Story--Taxol Crystals Masquerading as Microtubules

Andy Maloney, a Ph.D. student in our lab, recently read and summarized a very interesting paper in his open lab notebook. The paper, "Taxol Crystals Can Masquerade as Stabilized Microtubules," was published in PLoS ONE in January of 2008 by Margit Foss, Buck W. L. Wilcox, G. Bradley Alsop, and Dahong Zhang¹. Since our lab is now heavily involved in experiments involving kinesin and microtubules, and because it addresses something that had been a mystery to us, the paper really caught my interest. I'll explain more about that below. But before doing that, I wanted to talk about something probably of more general interest: a success story for publishing in PLoS.

Andy noticed that in their methods they defined BRB80 as having 4% glycerol. Glycerol is used to promote tubulin polymerization, and I've never seen it included in the BRB80 (aka PEM) definition. It could also affect solubility of Taxol, so it's an important detail whether or not a substantial amount of glycerol was in their standard BRB80 buffer. I strongly suspected that this was just an oversight by the authors...and I could easily have assumed this and moved on. But what about future readers of the article? Was there anyway to correct that article? For most journals today, even in the year 2009, the answer would have been, "no." However, this is no ordinary journal, this is PLoS ONE! All I had to do was select the text in question, and then click to add a note. After adding my note, an icon appeared in the article, allowing any future reader to see the question.


I don't know whether authors are notified when their article is commented on. (If not, it would be an important feature for PLoS to add.) So, I sent an email to the corresponding author of the paper (D. Zhang) pointing out the question. In less than a day, D. Zhang wrote back saying that he'd asked M. Foss to look into the issue. And then again in less than a day, Margit wrote me back to say that she'd looked at the original lab notes and indeed they'd made a bit of a typo in how they described BRB80 in their report. She added a very clear response to my note. She also went out of her way to point me to two subsequent papers that have extended their taxol microcrystal research^2,3. These authors deserve a lot of praise for responding to this question so quickly! A few months ago received a similarly rapid response from authors of another PLoS article...only two data points, but I wonder if PLoS authors are indeed more likely to respond quickly to questions from readers?

Now, why am I so happy and why do I think this is a success story for PLoS? It's because now, for the rest of time, when readers of this excellent paper do look into the methods, they will be able to see the corrected definition of the buffer used. Given how many times I've been burned by incomplete or incorrect methods, I do believe this will save substantial amount of time for at least a couple people down the road. (Will the PDF version of the article ever incorporate this note? As it stands now, I don't think it does...it would be very valuable if technology could be worked out to include links to these comments in future PDF downloads.) One more thing: I just noticed that Margit Foss today also posted a new comment on her article. She links to the two papers she'd told me about in her email, as "Relevant references on Taxol crystals." This is a great service to readers, especially since the newer reports^2,3 support a different mechanism for Taxol microcrystal / fluorescent tubulin binding. In summary, many thanks to PLoS for this wonderful journal and to these authors for their dedication to excellent science!

Now, if you're still reading, I'd like to also comment on the very interesting science in their report. Taxol (generic name is paclitaxel, I think) is a drug used in cancer chemotherapy. It's proposed mechanism of action is to inhibit mitosis by stabilizing microtubules in the spindle apparatus. In vitro, Taxol dramatically reduces the rate of microtubule depolymerization. Many people, including kinesin researchers in our lab, leverage this microtubule-stabilizing effect by adding Taxol to microtubule-containing solutions. What I learned from the Foss et al. paper is that the concentration of Taxol typically used in microtubule gliding assays (10-20 micromolar) is far above the solubility limit of Taxol (somewhere around 0.8 micromolar in aqueous solutions). Furthermore, they show that Taxol forms microcrystals above this solubility limit (even at 0.92 micromolar) and that often these microcrystals form a striking resemblence to microtubule bundles and asters! DIC images of these microcrystals (formed in absence of tublin) are shown in these images from Foss et al.¹:

(scale bar 10 microns)


The final piece of crucial information provided by this article is: these Taxol microcrystals rapidly bind fluorescently-labeled tubulin! (Later reports indicate that it's the fluorophore, not the tubulin that is binding to Taxol^2,3.) This means that many kinesin researchers (including me) likely have Taxol microcrystals in their samples, and because they become coated with fluorescent tubulin, there is a huge risk of misidentifying these structures as microtubule structures. Indeed, here is a recent fluorescence microscopy video that Andy took of something that at the time was a mystery but which we now know is likely a Taxol microcrystal decorated with rhodamine-labeled tubulin!

Likely Taxol microcrystal in kinesin / microtubule gliding motility assay (using rhodamine-labeled tubulin). Andy Maloney data.

In my past, I've also often seen these structures which I attributed to "clumpy" or "weird" microtubule structures. For example, I often noticed very bright, thick, and stick-like structures that I called "microtubule logs." It never occurred to me that they were Taxol crystals! (Also I remember that these structures were much less prone to photobleaching. I wonder if that's because (a) there are buried fluorophores inside the crystals, protected from oxygen, or (b) even on the surface of the crystals, Taxol somehow protects fluorophores from photobleaching?)

Foss et al., go further and speculate on whether this has important implications in vivo (i.e. in cancer chemotherapy). I can't really comment on that, but it's interesting to think about. What's most important for us is that we now know we have a problem with our buffers (too much Taxol!) and we may be able to fix it. The concentration of tubulin that we typically use is about 0.4 micromolar of tubulin dimers. Thus, for a 1:1 ratio of Taxol to tubulin dimers, we'd need 0.4 micromolar starting concentration of Taxol, which is below the solubility limit. There's at least two things I don't know: (a) What is the binding affinity of Taxol for microtubules? and (b) Do we need a 1:1 ratio to get significant stabilization? If the answer to (a) is something like a few nanomolar, then we may be OK with something around 0.5 micromolar (500 nanomolar) Taxol. If not, then we may have to hope the answer to (b) is "no."

A quick search just now yielded a paper from 1994 that says the binding constant for taxol to microtubules in 10 nM. That'd be good, except that they also seem to say that they only get stabilizing effects when the concentration is in the micromolar range⁴. Dang! Well, it shouldn't be too hard to try out 500 nM Taxol and to see whether MTs are reasonably stable. It's possible our MTs may be more stable than those used in the Caplow et al. study. It's also possible that the Taxol microcrystals are not affecting the kinesin activity in our system, and that we can do our studies at high Taxol concentration. Even if so, it's great to know about this issue so we can keep on the lookout for Taxol problems.

References

1. Foss M, Wilcox BWL, Alsop GB, Zhang D (2008) Taxol Crystals Can Masquerade as Stabilized Microtubules. PLoS ONE 3(1):e1476. doi:10.1371/journal.pone.0001476

2. Castro, J. S., Deymier, P. a., Trzaskowski, B., & Bucay, J. (2009). Heterogeneous and homogeneous nucleation of Taxol crystals in aqueous solutions and gels: Effect of tubulin proteins. Colloids and surfaces. B, Biointerfaces. doi: 10.1016/j.colsurfb.2009.10.033.

3. Castro, J. S., Trzaskowski, B., Deymier, P. a., Bucay, J., Adamowicz, L., Hoying, J. B., et al. (2009). Binding affinity of fluorochromes and fluorescent proteins to Taxol™ crystals. Materials Science and Engineering: C, 29(5), 1609-1615. doi: 10.1016/j.msec.2008.12.026

4. Caplow, M., Shanks, J., & Ruhlen, R. (1994). How taxol modulates microtubule disassembly. The Journal of biological chemistry, 269(38), 23399-402. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/7916343.

Foss M, Wilcox BW, Alsop GB, & Zhang D (2008). Taxol crystals can masquerade as stabilized microtubules. PloS one, 3 (1) PMID: 18213384

Link to FriendFeed discussion thread.

My first rating and commenting of a PLoS article in my own field (Scary!)

SJK 6/9/09: Here is a link to related friendfeed discussion.

I just finished reading and commenting on a PLoS One article that is near my own field of research. The article is titled, "Dissection of Kinesin's Processivity." The authors are: Sarah Adio, Johann Jaud, Bettina Ebbing, Matthias Rief, and Günther Woehlke. You can see my rating and overall comments here. (Since I'm not sure if that link will work, I'll also repost my comments below.)

Throughout the process of reading and commenting on this article, I learned a lot more about my fears and barriers to PLoS commenting. I discussed some of these in my prior post about my first PLoS rating. In contrast to my first rating, this article is smack in the middle of my field of interest (the kinesin molecular motor). I deliberately chose the most relevant PLoS article I could find. I'd estimate that my fear of placing comments was at least 10 times higher than for an article outside my field. I definitely felt like my comments were piping directly into the author's email inbox, ready to enrage them at any misunderstanding or criticism I posted. I still feel this way and am a bit worried. My worries are probably justified to some extent, since I am very new to this field. Thus, I could easily be seen as an ungrateful newcomer who hasn't paid his dues. And of course the people who wrote the article could end up anonymously reviewing my own papers and grants.

Given those worries, I came close to deciding not to post my rating. However after much reading and thinking about their results, I felt compelled to make a serious comment about error analysis supporting one of their conclusions (not their major conclusion). I was confident that my criticism was fair, and convinced myself that posting the comment was the right thing to do--perhaps I can save another reader a lot of time, or even help the authors out if they read it. I posted my criticism directly in the article, along with several typo corrections. After doing that (late last night), I realized that if / when the authors DO see my comments, they'll see a string of petty typo corrections and then this criticism, but nothing positive at all. That's a problem!!! Because of this, I decided to sleep on it, and compose an overall rating with positive comments today. I was busy most of the day, but finally tonight was able to finish my rating. In all honesty, though, without having travelled that slippery slope of commenting, I don't think I would have posted this rating tonight. I would have balked at the risk of angering the authors, sticking my neck out, and possibly being wrong. I probably would have convinced myself that these risks outweighed any meager potential gain that the world of science would get from my remarks.

I'm a bit worn out now. Hopefully in the comments here or more likely, on FriendFeed, we can talk about these things. I hope in the next couple days to expand on my review of the paper in my research blog, and to include it as my first Research Blogging attempt.

Reposting of my rating and overall comments on the article

This is what I submitted to PLoS as my rating:

Insight: 4 stars, Reliability 3 stars, Style, 4 stars.

The authors recently characterized NcKin3, which is the first known,
naturally dimeric but non-processive and plus-end motor. In this
report, they are leveraging this discovery to study chimeric constructs
between NcKin (a dimeric, processive Kinesin-1 motor in the same
organism) and NcKin3. They make two different chimeric constructs: one
with the head of NcKin and the neck of NcKin3, and the other with head
of NcKin3 and neck of NcKin. Importantly, the head included the core
motor domain AND the neck linker region.

I congratulate the
authors on a lot of very nice work that must have been very difficult!
The results they report come from an impressive array of difficult
assays spanning single-fluorophore position tracking, single-molecule
bead motility assays with optical tweezers, gliding assays, and a
variety of ensemble biochemical assays.

Study of the two
chimeric constructs, in comparison with the NcKin and NcKin3 wildtypes
allowed the authors to gain insight into which parts of the kinesin
motor are important for conferring processivity onto dimeric
constructs. (And also, which parts are important in NcKin3 for
inactivating one of the heads.) As far as I know, these are the very
first two chimeras created between these two kinesins and thus open the
door for many more investigations into how processivity is regulated in
the motor domain, neck-linker, and neck regions. The results here
indicate that many more chimeric structures and site-directed
mutagenesis studies will be necessary and valuable. Of course, that is
a lot of work, but the results here open the door for those further
studies.

For me, the most fascinating result was point (iii) on
page 4. The authors show that the Head3/Neck1 construct seems to get
stuck in a "kinetic dead end." As they say, the kinesin-1 neck appears
to confer some elements of processivity, but not all. Combined with the
missing elements (which kinesin-3 head lacks), the motor is actually a
bit more handicapped, as shown by a gradual decrease in gliding
velocity as the concentration of motors is increased.

I also had a couple questions about the paper that I noted previously (see prior article comments):

* Statistical significance of processivity measurements.

* Lack of discussion and comparison with previous Ncd/Kinesin-1 chimera results

DISCLOSURE:
Our lab (http://openwetware.org/wiki/Koch_Lab) has recently obtained
major funding to study kinesin. I do not think we have competing
interests with these authors or the work they've presented here, but I
thought it worth mentioning.

My first PLoS comment: High rating of an article on TSLP being the cytokine link between eczema and asthma

5/27/2009 SJK Note: After I wrote this, Bora Zivkovic sent me links to the PLoS community blog where he talks about commenting and rating PLoS articles. Both are very much worth reading! Bora is the Online Discussion Expert for PLoS.

Recently, William Gunn Mr. Gunn composed an excellent article discussing online identity and the making of public comments in scientific circles. Without immediately spiraling into a stream of ridiculous conversation, I can't really comment on his post, or the ensuing friendfeed thread. Suffice to say that Mr. Gunn and others on friendfeed inspired me to be a lot bolder in commenting on PLoS articles.

So, tonight I made my first comment on a PLoS article. Previously, I had viewed commenting on the actual article site as a very formal procedure that required attaining the highest level of understanding of the article before submitting a comment. Essentially, I was viewing commenting on an online article the same way I viewed submitting an official comment to an article published in Science or Nature (or other journals). Published comments in those journals are almost always refutations of the article that seemingly without fail lead to concomitantly published rebuttals by the original article authors. Thus, the culture of commenting on articles is fraught with nastiness and putting one's scientific reputation on the line. This could be the reason that so far "official" online commenting on peer-reviewed articles has been very limited, whereas "unofficial" or off-site commenting has been more common. By "unofficial," I am loosely referring to comments made anywhere that is at least one link removed from the actual published article site. For example, an external blog, friendfeed discussion, or notes left on article managing services such as citeulike.

It occurred to me while laughing and crying my way through the recent friendfeed discussions (OK, fine, here's a link to perpetuate the madness) that this culture may be relatively easy to change. (Aside from any questions of whether it's necessary to change.) In my opinion, PLoS has already made one innovation that vastly increases the odds of a user making a public "comment." They have separated the article ratings into three categories: Insight, Reliability, and Style. From my personal experience, that opens the door almost all the way in terms of inviting some kind of reader feedback. Rating an article on "Style" does not carry much professional risk from my viewpoint. Rating on "Insight" requires understanding of the possible impact of the article, and is thus much more weighty than the "Style" rating. However, I personally feel I can rate an article on "Insight" without assessing the quality or reliability of the methods and data. I recently did this with a PLoS ONE article I saw on single-cell sequencing of uncultured organisms. To rate an article on "Reliability," I feel requires the kind of in-depth understanding that would be required for me to send a formal letter into the editor of Science or Nature that could be published. Thus, the barrier for me to rate on "Reliability" is quite high. Especially since if I'm going to put in enough effort to feel completely justified in rating, it's likely to be less than a 5-star rating. (I guess I'm feeling like I spend more time reading articles that I disbelieve than those I do believe?)

Another reason that placing online comments does not have to be as formal and negative as with traditional published comments is that the comments are published without a delay waiting for the original authors to compose a response. This then reduces the expectation that the publishing authors must respond and therefore takes the formality down a bunch of notches in my opinion. Also, in terms of PLoS the whole mission of the journal is to make research more broadly and rapidly available--and thus I think there is an expectation that the comments should also come from a broader base of readers.

So, that is what inspired me to take the time to read a PLoS Biology article and compose my first online comment tonight. I was also inspired by the belief that we're still very early in the process of dictating the culture of online discussions of peer-reviewed research--and thus a concerted effort can make impact in what ends up happening. This inspiration was combined with the coincidence that my wife sent me an article from BabyCenter today that caught my interest because it was discussing the recent PLoS Biology article. Finally, the thing that finally tipped the balance and convinced me to take the leap and make my first PLoS comment was a healthy dose of "WTF" So I stopped worrying and took the leap. :)

The value of an open-access publication record for an academic job search and tenure & promotion.

11:22 am update

Steve says: I just read Gideon Burton's excellent post about "Intellectual Apartheid." One of his recommended steps for administrators is "Update promotion and tenure policies to favor open access publications and to accommodate evolving scholarly genres (such as data sets, software, and scholarly tools that build the cyberinfrastructure)."

Earlier this week, my department chair sent our department a link to an article in The Chronicle of Higher Education about MIT's open-access policy announcement. (I believe there is the standard irony that the article from the Chronicle is limited-access, but you may be able to find freely available stories on Google news.)

Without being an expert on open-access or doing much background research, I decided to send the following email to my department. I'll let you know what happens (if anything)!

Email to Physics & Astronomy faculty:

This got me to thinking. Our department could adopt a simple & public policy, such as: "Regarding new faculty hires and promotion & tenure decisions, we highly value an open access publication record. We place a value on open access publishing comparable to the value we place on publishing in top-tier scholarly journals which may have limited access." I don't know whether we could agree on such a statement, but if we could, I think it would place a positive light on our department, similar to how the MIT and Harvard statements below do for those universities.

As a tenure-track faculty in our department, I do feel that open-access publishing will be viewed positively by the voting faculty. It would be good to know that more formally, but I'm not worried. A much more worrisome thing for me is how open-access publishing will affect my Ph.D. students.
Will they lose out in job searches or will they stand out? Our own department's stand on this issue won't help our own students. But maybe by taking a public stand, we can set an example that other departments can follow.

I think it'd be worth spending a bit of time discussing at an upcoming faculty meeting.

--Steve