The inevitable spread of open science

Two things have happened this week that make me really happy about the research in our lab and the spread of open science.  First, we have a new undergraduate REU student, Alex Haddad, who has started her own open notebook science under the mentorship of Anthony Salvagno.  Her notebook is on wordpress.com and can be found here.  This is Alex's first experience in a research lab and she has immediately embraced open notebook science and she is excited about it.  One cool thing that I've noticed already is that her notebook entries are automatically linked in Anthony's notebook when she links to them.  Some kind of trackback thingy that I don't understand, but is great as far as good notebooks go.  An example can be found in Anthony's notebook entry, which automatically links to Alex's entry providing more information (see the trackback at the bottom of the page).  Welcome, Alex, to open notebook science!

The second thing that happened is that our former PhD student, Andy Maloney, just started a new postdoc at UT-Austin with Hugh Smyth.  This is going to be a very productive experience for both Andy and Hugh's lab, I am confident.  Most excitingly, though, is that Andy and Hugh have decided to incorporate open science into their projects!  I think this is very big news and a success for the spread of open science.  Major props to both Andy and Hugh for their willingness to carry out major parts of their research using open science!  I had some further thoughts on this and the implications for the spread of open science.  Instead of re-writing them, I'll just quote my comments on the FriendFeed thread:

I think big factors are Andy's commitment to open science and his new PI's commitment to making an impact in science and medicine.  I met Hugh Smyth a few times when he was at UNM and only detected awesomeness, both in his research and in his mentoring and concern for students.  Openness is probably going to be more challenging for them, though.  One reason is their research is much more applied and medical, and thus IP plays a major role.  The field is probably a lot more competitive.  And their lab is much more successful with funding.  As Nielsen and others have pointed out, the current reward system stacks the cards against openness.  So they will have to be careful.  But I think they're clever enough to figure out how to do it, and their success will pave a lot of roads for future openness.  I've been thinking about it pseudo-mathematically and I think the fact that they're even willing to try is a success.  I've had two PhD students graduate so far.  One is likely in industry for a long time and unlikely to be open for a long time if ever.  The other, Andy, is now at least partially doing open science.  The subsequent students in our lab (Anthony, Alex, Nadia, Pranav) are still performing open science.  A former intern, Diego Ramallo Pardo is in grad school at Stanford and has a passion for openness, but not able to be open yet.  Dozens of undergraduate lab students have performed open notebook science in my lab course, and there have been a few instances of continuing ONS after the course (most do not continue in research careers).  So, at first glance it appears that there isn't a high rate of spread of openness from our research and teaching labs.  But it occurs to me that it doesn't matter.  If we were to model openness as an infection, it's a powerful one.  I think it's even a latent infection in almost all scientists.  Participating in openness awakens the infection for life and it sheds constantly.  The immune reaction is our current system of practicing and rewarding science and it's quite powerful.  So it wins in a lot of cases.  Nevertheless, openness is slowly winning more often and the immune system is not going to adapt to get stronger.  On the contrary, the immune system is going to take major hits in the coming years.  Funding agencies are going to change rules.  Tenure and Promotion and hiring committees are going to add members who value openness.  Closed-access publishing for profit is going to topple precipitously.  And at that point, openness will spread and emerge naturally and quickly.  It seems plain as day to me.  Now, one of you all can translate that into epidemiological mathematics and fiddle with some exponents.

Open Access Week event at U. Arizona: Reproducibility, Open Data

Earlier this week I was lucky to participate in the Open Access Week event at the University of Arizona: The Future of Data: Open Access and Reproducibility.  The event was hosted by Chris Kollen and Dan Lee of Arizona University Libraries.  I am very grateful for the invite and the opportunity to meet them, some active member of the audience, and the other speakers, Victoria Stodden and Eric Kansa.

Victoria Stodden gave an excellent talk, framed around the computational sciences, and with the major point: Instead of promoting "open data," we should promote "reproducibility" in science.  She argued, very convincingly, that good science requires reproducibility and thus scientists should be easily convinced that we need very high standards for reproducible results.  For computational research, the only way to ensure reproducibility is to publish much more open data and open code than is normally done now.  If your result is computational, how can anyone hope to replicate and build upon your results if you haven't provided the source code and the data sets?  They can't, but publications without code and data are by far the most common these days.  It's a failure of science that is probably caused by many factors.  One that comes to mind is that computational scientists have been forced to fit their "publications" into standard peer-reviewed articles, where the system is not set up to accept and / or host source code and data.  (As an aside, this is clearly a routine failure of peer review, as referees obviously are not ensuring reproducibility of the research, which should be a primary criterion for publication.)  Scientists understand that reproducibility is an essential element of research.  For example, two years in a row, my undergraduate physics majors identified reproducibility as the most important element of good science (see brainstorming 2010).  Since scientists understand this, then they will naturally practice open publishing of data, code, methods when they realize that reproducibility is missing without those elements.  As Victoria argued, demanding "open data" leads to confusion and resistance and ultimately probably lack of compliance.  In contrast, demanding "reproducible research" is already a cultural norm and it naturally leads to open data and open code of the most helpful variety for reproducibility.  Victoria's slides can be found here.

The notes for my presentation can be found on linked mindmaps, starting here.  (Click on the tiny right arrows to navigate.)  My notes are probably not too meaningful if you weren't at the symposium.  In contrast to Victoria's high-level talk about policies that could make a major impact, I told a few stories about open data and open notebook science in our own teaching and research labs, and the successful impact we've had already.  I think (hope) it provided concrete example of the benefits of open science.  On the one hand, I showed that open science, especially open notebook science strongly promotes reproducibility.  This has been seen best in the undergraduate physics lab that I teach.  Students read the notebooks of other students from prior weeks and prior years.  They build upon these previous results, which allows them to get the experiment working much quicker and have more time to explore new aspects of the experiment, or to develop new data analysis methods.  They are doing real science!  I showed an example of an excellent primary notebook from Alex Andrego and Anastasia Ierides.  However, I think I also showed that open data and open science make an impact beyond just reproducibility.  This impact is in reuse and repurpose of data. I told two stories where theory and research groups already have been able to use data we publicly shared on youtube.  One group has already used our data in a theory preprint on the arXiv.  Both groups expressed delight and gratitude that our data was freely availalbe.  There are two important features of these stories.  First, both groups used our data for a purpose that we had not (and probably would not have) imagined!  Clearly the impact of our data was multiplied by being public.  Secondly, we did the easiest and simplest sharing method we could find: youtube, yet we still made an impact.  We are currently working with Rob Olendorf, a data curation librarian at UNM to vastly improve our sharing.  This will include permanent citation links, vastly improved metadata (at least 10x more than the data itself), hosting by the institutional repository (much safer than our lab server), and links to other data sets.  Reason would have it that if we could make an impact with the imperfect system we tried first, then the impact will be much higher with the data shared via Rob and the institutional repository.

The final talk was by Eric Kansa, who described the amazing work of him and his colleagues on Open Context, a platform for sharing and linking archaeological data.  His notes from the event can be found here.  And his slides are available also: A More Open Future for the Past.  Despite being far from the field of archaeology, it was easy for me to see the vast impact that Eric and his colleagues are making via the open context project.  A large amount of time, sweat, and money are expended collecting archaeological data.  Without opening these data and curating and linking these data, the potential impact is severely limited.  The Open Context team has developed a method for collecting these data, archiving them, and linking them to other data sets.  The method is very effective, and importantly requires far less work than required to collect the data in the first place.  This seemed clearly, to me, a case of the huge power of data reuse and repurpose. In contrast to computational science, the power of data reuse seemed to trump the need for open data for reproducibility.  This is not surprising, given how different the two fields are.  But it was an interesting and somewhat confusing contrast for me between the needs for open data in computational research versus archaeology.

There were several engaged audience members.  One of them was Nirav Merchant, with the iPlant Collaborative.  Victoria and I were highly impressed by the computational platform that iPlant has developed already, only three years into the NSF cyberinfrastructure project.  I was simply amazed and I couldn't do it justice describing it.  The ability to ensure reproducibility of computational research with the iPlant platform is vast.  One example is how easy it is to save an image of a virtual machine and then share this image with other users.  They demonstrated this for us and it took only a few clicks and less than a minute.  I highly recommend reading more about iPlant at their site linked above.  The iPlant team that we met was energized, engaged, and collectively brilliant.  I'd love to know how they assembled their team as they've clearly done an excellent job.  I intend to keep in contact with the iPlant folks and am even hoping that I could introduce the computational platform to my Junior Lab students this year.  I think the exposure to these state of the art and "open" tools will be invaluable for their future research.

Overall, the one-day Open Access Week event was highly successful for me.  I met some amazing people and gained a lot of clarity in my thinking about the imperative for much more openness and sharing in science. Incidentally, maybe not coincidentally, during my flights I was able to read Michael Nielsen's fantastic new book on the untapped potential of connected, open science: Reinventing Discovery.  Despite having met Michael and having heard him speak a few times, I still found the book riveting and I learned a lot.  I absolutely recommend the book to anyone interested in the practice of science!

An open data success story

Over the winter break, Andy Maloney and our lab enjoyed an open data success story.  Andy shares his data publicly with a CC0 / public domain license.  Some scientists ran across the data, I think by Google searching and contacted us to ask if they could use our data to support their research.  Since it is CC0, they didn't have to ask, but like most scientists, they were courteous and did contact us.  I shared this story at the ScienceOnline2011 "Data Discoverability: Institutional Support Strategies" session and I think people liked the story.  Jean-Claude Bradley mentioned it in his blog summary of the conference, and Lucy Power saw this and contacted me for more details.  Lucy's is studying e-Research for her Ph.D. dissertation topic.  I sent her a reply, and instead of rewording it, I will just past it below.  I can answer questions on the FriendFeed thread.  Yay Open Data!

Hi Lucy – I definitely should write up a blog post about it and I will try to do that soon.  I think it’s a great little success story for open data and data reuse.  In a nutshell (and I can answer questions): Some people found Andy’s microtubule gliding assay data on youtube and emailed us to say it was very interesting to their theoretical work and could they use our data in a pre-print.  We replied “of course!” “woo hoo!” and we told them that it’s all public domain data so they are free to do whatever.  As a courtesy, we said we’d like a shout-out.  They went further and offered co-authorship, but Andy and I decided an acknowledgment was more appropriate at this time.  Andy suggested they acknowledge open notebook science, etc. and they did in their pre-print.  You can find the pre-print here: http://arxiv.org/PS_cache/arxiv/pdf/1101/1101.2225v1.pdf see Figure 3A for Andy’s data and the acknowledgments section.
 I think it’s a great success story because (A) they never would have known about our data if it weren’t open.  It didn’t necessarily have to have an open license, but it needed to be discoverable.  (B) we never would have thought to use our data for this purpose.  So obviously value was created via openness. OK, I’ll try to write up the story in a blog or something soon!  (Maybe I should just post the above and not worry about wording it better? J )
 --Steve

FriendFeed Thread:

Thank you, Addgene, for the award!

A big "Thank You!" to Addgene for giving Andy Maloney and our lab a "Resource Sharing Award!"  The award is a $5,000 donation to our lab that we can use to further our kinesin research.  Very generous and very helpful to our lab.  Big props also to Andy for applying for the award with no help from me!  One more piece of evidence that the students in the lab are much better at grant writing than I am :) 

The award was given to Andy and our lab for our commitment to open science.  This includes open notebook science, open data, sharing protocols, designs, etc.  Andy has been a very impressive open scientist.  It's just a guess, but I'd say so far, probably his biggest impact has been with the very detailed "do it yourself" biology projects he's contributed.  He's absolutely amazing with designing solutions from off-the-shelf components, and equally amazing with using Google Sketchup and photographs to describe the designs to the public. A good example is his microscope objective heater, which was somewhere around $500 and is working very well for our gliding motility assays.

There are now many labs around the world deserving of this award, and it feels really good to receive it.  And I think it was a great contest for Addgene to sponsor.  I actually wasn't aware of Addgene before Andy told me about the contest.  So just learning about them made the effort worthwhile.  I had a great conversation with Melanie Herscovitch on the phone a few weeks ago and she explained to me Addgene's mission and services.  Here's a picture from their website to explain what they do: (used without permission! :) )

In a nutshell, addgene is a non-profit organization dedicated to making it easier for researchers to share and obtain published plasmids.  Authors of papers submit their plasmids to Addgene (either purified DNA or transfected cells, as I understand it).  Readers who would like to obtain the plasmid contact Addgene, and Addgene provides the plasmids for just a cost-recovery fee.  This works out well for all parties.  Without Addgene, it's often a very inconvenient process.  The authors are burdened with keeping track of plasmids that may have fallen out of use.  And researchers requesting the plasmids often face a long delay in obtaining them.  I think Addgene is a wonderful service and I look forward to working with them as we create and publish our own plasmids in the coming years.  I also got the feeling from talking with Melanie that Addgene is a really great place to work.  I don't know whether or how often they're hiring, but you can take a look here for current job openings.

Thanks again, Addgene!  If you're reading this, it'd be great to post a little thank you comment (on friendfeed or the blog) or congratulations to Andy!

FriendFeed Thread:

Science Commons Symposium Pacific Northwest 2010, quick summary #scspn

Today I was lucky to attend the amazing Science Commons Symposium.  There were back-to-back fascinating presentations by Cameron Neylon, Jean-Claude Bradley, Antony Williams, Peter Murray-Rust, Heather Joseph, Stephen Friend, Peter Binfield, and John Wilbanks.  It was wonderful to meet in person a few people that I either did not know before, or whom I'd only know online previously, including Lisa Green (who went out of her way to invite me to attend this conference, thank you!), Heather Piwowar, Anali Perry, and Brian Westra.  It was also a great pleasure to meet again people whom (with the exception of Cameron) I'd only met in person a few weeks ago at the ScienceOnline2010 conference: Jean-Claude, Cameron, Hope Leman (another tireless organizer of the conference who graciously invited me to attend), Bill Hooker, Pete Binfield, and Antony Williams.  (My apologies if I missed out on name-dropping anyone, it hurts me more than you!)

I'm off to the Biophysical Society meeting tomorrow morning, and a bit out of steam, so I'm going to cop-out a bit and just embed a mindmap of my notes from the meeting.  Before doing that, I'll post a few action items from the meeting, and maybe later I'll come back and link to specific friendfeed or other threads for the items:

• Antony and I made some progress discussing our athletic challenge to raise money for asthma research or other charity.  I think it's promising we can make it "generative," and successful.
• Pete Binfield used Heather Piwowar's PLoS ONE paper as an example.  I want to read it and then rate it.
• Improve our lab's Open Notebook Science.  This is ill-defined, but there are many steps we can begin taking immediately to work towards a system that works for us as well as it does for Jean-Claude Bradley and his students / collaborators.

OK, Here's the mindmap.  I started doing it as an example for Heather, Hope, Bill and others from a discussion at dinner.  Too tired to convert it into regular text now...would be interested to know if it's useful at all to you!

FriendFeed comment thread:

ScienceOnline2010 -- Top N things I learned and was inspired to do at #scio10

• FriendFeed thread for this post
  

I had a blast at ScienceOnline2010 last weekend!  Thank you Anton, Bora and others who spent so much energy organizing it!  Approximately 250 people attended and it was a very diverse crowd of scientists, science writers, publishers, librarians, science outreach specialists, high school teachers, even high school students.  Much has already been blogged about the conference, including many "Top N" lists.  You can find a list of them here.  My favorite so far is Jonathan Eisen's "Enough w/ the good: here are the top10 problems w/ the #scio10 meeting."  Hilarious!  Despite everything having already been done, I hereby present my belated list of things I learned, things I did, and things I've been inspired to do:

1.  I had a ton of fun interacting with a bunch of e-friends, old and new.
I attended as a scientist with an interest in all of the other areas.  It was definitely a new feeling to be in a session and have the speaker ask, "how many of you are scientists?" with the answer being a very small fraction of the participants in the room.  It was also a very new and thrilling experience to finally meet in person many people I've known only virtually for the past year.  I think Cameron Neylon was the only person I'd met in person previously.  Despite that fact, it was incredibly easy to have conversations during lunch, between sessions, and of course at the bar.  I uniformly enjoyed these people even more in person.  An incomplete list of the people I had the pleasure of chatting with include: Bill Hooker, Jean-Claude Bradley, Pawel Szczesny, Walter Jessen,  Christina Pikas, Hope Lehman, Peter Binfield, ...  Plus I found some completely new friends at the conference, including Antony Williams, Greta Munger, Dorothea Salo, Andy Farke, Natalie Villalobos, Michael Habib, Annie Crawley, ...  Now, those previous two sentences could be perceived as egregious name dropping, which I am guilty of, simply because it's quite an amazing list of people, none of whom I knew before I became active in open science.  Clicking through those names and reading what they're saying, and you'll realize why I feel so lucky to have met them!
(Note, I forgot: Fabiana Kubke, ...)

2.  Antony Williams and I made some kind of running challenge.
I knew of Antony Williams from his ChemSpider fame.  I also had recently read one of his personal blog entries, about running 1000 miles in a year, along with his ugly and unfortunate calf injury.  I'm not sure I knew these were the same person, though.  Nevertheless, he walked passed me in the hotel bar, and I accosted him to inquire about his calf injury.  I'm pretty sure he didn't know me at all, but luckily we had on slick name badges and I was surrounded by credible people.  It'd be a big challenge for me to recount the conversation (I swear I remember it perfectly, I just don't feel like writing it down).  Let's just say that I was happy to learn that he'll be back running again within a week or so and that he has a goal of raising money to fight asthma.  I have had an idea mulling in my head that I could raise money and get motivated to get better at running by setting a race time goal.  I thought this was a perfect match with Antony's goal, so I quickly challenged him to a running competition.  He quickly agreed (fearlessly) and I tweeted/friendfeeded it to lock in the deal.  Over the next week, I'll see if I can clarify the challenge and I'll post updates to that thread.  I'm thinking I'll setup a Google spreadsheet for me and others to place their pledges and monitor the progress.  Antony already knows about Nike+ technology, and I'm looking forward to doing something like that too.  It'd be a good way to try out new things in open data, and open notebook science, actually.  Suffice to say that I'm going to get better at running, lose a lot of weight, and hopefully we'll raise some money too!

3.  I and KochLab are going to get better at doing Open Notebook Science and sharing data & software this year
I learned a lot at the conference about tools that exist for carrying out open notebook science and sharing data, methods, software, etc.  I still have a lot to learn, and indeed many tools still need to be developed.  But I know that our lab can make improvements this year.  Here's some concrete things that we'll do:

• I've sent an email to Amy Jackson, Digital Initiatives Librarian at UNM, requesting a meeting.  I had briefly spoken to her via email in October and now I'm fully energized to have a meeting with her and see what kinds of first steps we can take towards building a partnership between the library and our lab in terms of sharing data and conducting open science.  I'll try to leave updates on this FriendFeed thread.
• Get better at sharing software.  Currently we use LabVIEW, which is a wonderful programming environment.  One of the major benefits is that it's a graphical, data flow language.  This makes the code a 2-dimensional diagram...so in my opinion, it's exceptionally easy to read other people's code.  Unfortunately, it's a proprietary and expensive coding environment.  You are allowed to compile .exe and .dll files for others to use freely.  But that's not open source.  So, there are two routes to go: (1) We could compile virtual machines (VM) and send those to people for exploring our code.  For example, referees of papers we submit.  This was an idea from Deepak Singh at the meeting. Licensing is an issue here, and what I'd like to do is find someone at National Instruments (creators of LabVIEW) and discuss what can be done to serve our open source needs.  (2)  Learn a text-based, freely available language.  I think this would be valuable for our students anyway, in terms of building their resumes.  While at the meeting, I thought Ruby was a good idea, but now not so sure.  I've posted a FriendFeed message about this, and have received all kinds of very valuable advice.
• Adopt techniques to make it easier to capture our workflow in the lab.  OpenWetWare has innovations coming up soon and we'll certainly jump on those.  Cameron also hinted at something revolutionary coming up this year, but said he'd have to kill Bill if he told him what it is.  It wasn't clear that he'd have had to have killed others in the room, so I was disappointed he didn't say what it was.  (Just kidding, Bill!)   But the fact is, I don't think our current tools are nearly sufficient.  I'll put in more effort to make positive steps here, but I'm not sure exactly what yet.

4.  Improve the state of publishing, one article at a time: Try out some ideas via PLoS ONE.
I was delighted to have the opportunity to talk with Peter Binfield at the bar and discuss publishing ideas with him.  I have a lot of ideas that I'd like to try out, and it hasn't yet been proven that all of them are lousy.  I ran many of them past Peter and his general response was, PLoS ONE would love that -- it's just that whatever scientific community you're in may not respect it.  Very true.  However, I do think many in my community would respect anything that showed how publishing could be better by being different.  And I now realize that PLoS ONE is a very good platform for trying out a few things.

• At lunch with Bill and Pawel, they mentioned an idea that others had been talking about.  Unfortunately, I don't know to whom to attribute the idea. (Note: Bill and Pawel give attribution to Fabiana Kubke, which sounds right to me now.)  The idea is that publishing one very good figure would be a good idea.  I like this idea enough to think about trying it out.  It is related to an idea Larry Herskowitz and I were discussing a few days before the conference:  Can we publish a paper without an introduction?  Or an introduction that just says, "see such-and-such other paper for introduction?"  So much time is wasted rewriting introductions (in my opinion), especially when trying to avoid copying prior written work.  Is this necessary?  Publishing a single figure takes this even further.  I have some interesting data from grad school and postdoc that I have not been able to publish.  It pains me that it's just sitting around, useless.  Do you think it's worth using this data to test out the "publish one figure" method?   If I make progress on this, I'll post updates on this FriendFeed thread.  (Note added: Cameron Neylon reminded me that BMC Research Notes may be a better venue than PLoS ONE for this.)
  
• For maybe ten years now, I've thought that the value of anonymous peer review is overstated.  During that time, I've heard other people, much more eloquent than me also express this opinion.  Just briefly, I think fully-attributed, non-anonymous peer review would solve many problems that exist with today's science, and I discussed this a bit at the meeting.  Two of these problems are: (a) good referee work is difficult, and good referees are not credited for the work and the original ideas, and (b) a whole lot of incomplete and sloppy work is submitted and much of it is published due to ineffective referees / editors.  The solution I like is for every aspect of the peer-review process to be published, including the original manuscript submitted, all subsequent revisions, and all communication between authors, editor, and referees.  Clearly this solves problem (a).  As for problem (b), people often say, "but people aren't going to say negative things if their name is on it!"  First of all, that's not necessarily true.  Secondly, referees have the option to decline without comment.  If the editor cannot find someone willing to slam the paper, then it's just returned to the authors.  Voila!  One less crappy paper published.  The arguments get more complicated, especially when considering that grant review is anonymous, providing a lot of opportunity for underhanded retaliation.  I can't mathematically prove that it's a good or bad idea, so I'd like to try it out and see what happens.  It occurs to me that I could submit a manuscript to PLoS ONE and request that the Academic Editor implement this idea.  Why not?  Should be OK as far as I understand the rules.  I don't expect to submit garbage, so it won't be a complete experiment.  But something may be learned, and at least all the referees will get credit.  I may update progress on this idea on this FriendFeed thread.

OK, That's enough for now.  I can always add more later, especially since I aptly named this post.  For example, I may talk about the rebranding of our blogs that Walter Jessen recommended.  It's a good and valid suggestion, but I'm not adding it yet, because I'm not sure how soon I'll be able to think about that :)

Pondering our Kochlab graduate student compass...how about "Always Contribute?"

Last weekend, my friend Richard Yeh posted a couple essays by Paul Graham onto Facebook. I loved the essays and linked to one of them on friendfeed. Michael Nielsen, in turn pointed me to another essay by Graham that he thought I'd like, "How to Do What You Love." Michael was completely right, I loved the essay. If you have not read that essay, I command you to stop reading my blog and to go read that article! You'll get much more out of his essay than this blog.

OK, now that I have you defiantly reading my blog, intent on garnering something useful from it, to spite me, let me continue. The "Do What You Love" essay resonated with me very strongly. It reminded me of the discussion of talents in "First Break All the Rules" by Buckingham and Coffman. I think Graham and Buckingham and Coffman are talking about the same thing: that finding work you love is a key to happiness (and productivity), but that finding out what you love is a very difficult task worth working very hard on. The language of Buckingham and Coffman is to talk about finding one's "talents." I've been talking with my graduate students about this a lot for the past six months. (In fact, it's time for me to have another awkward talent-finding session with them, I do believe!) I also preach to all of my undergraduate students about the importance of finding their talents and I give them an end-of-semester assignment to think about their talents. I'm delighted to have been shown the Graham essay, because I think it is yet another way of presenting this argument to students, and a very eloquent one.

Since I loved the essay so much, I sent it to the person who gave me the First Break All the Rules book. He wrote back to me and keyed in on the "always produce" part of the Graham essay:

"Always produce" is also a heuristic for finding the work you love. If you subject yourself to that constraint, it will automatically push you away from things you think you're supposed to work on, toward things you actually like. "Always produce" will discover your life's work the way water, with the aid of gravity, finds the hole in your roof.

While reading the note, the Do What You Love essay finally clicked with another I read by Graham last weekend, "How to Make Wealth."[1] It's another fantastic essay that I feel like commanding you to read. One premise in that essay is that people in start-up companies can be 20-30 times more productive than they can in an ordinary 9 to 5 job. Thus, a small group of people can create a tremendous amount of wealth by working really hard for a few years. They can also get financially rich as a reward for their production of wealth for the world. The thing that clicked for me is that you cannot make the world a better place without producing. Most people are producing at a rate at least 20 times less than they could be producing, if they found what they loved and were able to do it all the time. I garner great optimism from this fact that we're on average so incredibly inefficient. It means most people are not even close to any absolute point of diminishing returns, and with the right kinds of changes, they could easily multiply their productivity and impact on the world by manyfold.

So, then I started thinking about our research lab and the students in our lab. I thought over things that each student has done in the past year that made me profoundly happy. As I thought over all these things, I realized they had a common theme: I was recalling instances of those students being unusually productive. Furthermore, my favorite recollections involved those where the students had shared their work on our public wiki, or our blog, or in some other fashion open to the world. This made me think that it is now very easy for me to summarize my main expectation and goal for my students: "always produce," borrowed from Paul Graham, of course. My students like to make Kochlab slogans, so I thought of "Kochlab: Produce" or "Kochlab: Always Produce," but if you pronounce the "Koch" correctly ("Cook"), then it has the problem of making one think of produce the noun, e.g. apples and bananas. Thus, I am thinking something like "Kochlab: Contribute" or "Kochlab: Always Contribute."

In some sense, "always contribute" means the same thing as "always produce." The point of the producing, in regards to Graham's essays is that you're creating wealth, and therefore contributing something to society. However, I like "contribute" much better, because it has much more clarity in the science world. "Contribute" automatically points the way towards open science (aka Science 2.0). Whereas, production in the traditional scientific world ("closed science") can be done with a very limited amount of contribution.

I have mulled it over for a couple days now, and I think I really like this as the main piece of advice and constant guidance to give to our students: "always contribute." Does this work? Let me try it out in a few ways:

1. By the time the students get their Ph.D.s, I want them to have learned a tremendous amount about what their talents are. I want them to clearly see what the next step in their career should be in order to leverage those talents and help them be successful and happy. A compass of "always contribute" will lead the students towards finding ways of being productive instead of spinning their wheels. These activities will be the means by which the students and I discover what their talents are. This is the point of Graham's "always produce" advice. Check.

2. By the time the students get their Ph.D.s, I want them to have a strong and large professional network of people that know them and the work they have done. "Always contribute" tells them that Open Notebook Science is a good thing to do. Sharing code, design drawings, personal summaries of research papers, tips and tricks on protocols -- these are all ways to contribute. In our limited experience in our lab, we have received validation after validation after validation that open contributions get attention. We can see this vaguely via page views or Google search rank or quite vividly via positive feedback from people that we admire and people whom we've helped. Combined with traditional publishing (also a contribution) and attending scientific meetings (contributing), I think "always contribute" will make building a powerful professional network almost automatic. Check.

3. I want our lab to produce innovative, exciting, and high-impact scientific results. Will "always contribute" point us in the right direction for this goal? Does it point in any direction? I need to think about this one some more. I feel like it must point in the right direction--for example, innovations are contributions. But there's some risk that focusing on contributing could lead towards a lack in overall production. Basically I am thinking of the standard arguments against open science -- increased likelihood of scooping, which in turn reduces chances of funding and publishing. I fundamentally believe that those arguments are strong enough to tip the balance, but I don't think they've been proven yet. Another example of how "always contribute" may be counter to our lab's scientific productivity: some students may discover that they are wickedly talented at contributing in ways that do not advance their research projects. That's a great thing to discover! An example that hasn't happened in our lab yet would be for a student to discover that they're fantastically talented at writing popular science articles and want to do so at the expense of doing any research. I want students to discover something like this. It fits perfectly with items 1 and 2 above. However, it is clearly a problem in regards to item 3. This is not a new problem, though. My job as a research professor is to both mentor students as well as ensure production of research results. The way the system is set up those goals are not always aligned, and sometimes in conflict. It's possible to be rewarded with research grants, even by abandoning the best interests of your graduate students. Most people in the system know this and have seen the devastating results it has for too many Ph.D. students. I am absolutely against doing that and despise many people who have chosen that route. On the other hand, I don't have a good idea about what do do if "always contribute" turns into "I can't do my research." That's definitely going to happen eventually. In many cases, it will be possible for the student to discover their true calling in life, but then re-focus on making the research contributions necessary to finish the Ph.D. that they've invested so much time in. Will there come a time when the student should rightly choose abandoning the Ph.D.? Ugh, this is a tough one: Item #3 gets an: almost check / need more thinking.

I've painted myself into a corner now. If I were Paul Graham, I would figure out a way to backtrack. But I'm not, and I don't really know how to end this blog post, so I think I'm going to end it by linking to another Graham essay about writing essays. This one was linked to me by Kartik Agaram on friendfeed. It's an essay that explains why high school and college writing assignments sucked so badly. If you hated those assignments but never quite knew why, you'll love this story. Plus, you'll feel vindicated and it will give you one more reason to trust your gut in the future. For example, if your gut were telling you that "always contribute" is a fantastic compass to present your graduate student mentees.

Footnote:
[1] I'm taking a bunch of liberty here with my own story. It wasn't until I started writing this blog that I realized the two essays had clicked. But I think subconsciously this is what was happening. Also, I probably had the Gin, Television, and Social Surplus essay by Clay Shirky in my head, as Joelle Nebbe had linked to it recently.

Personal open science challenges

There was recently a very interesting thread regarding open notebook science in the Science 2.0 friendfeed room. This was in response to Michael Nielson announcing that Tobias Osborne had begun doing open notebook quantum information theory. I think this is fantastic, and my kudos go to Tobias (whom I don't know). The friendfeed debate had to do with whether Tobias's work can be called open notebook science, which has a specific definition.

The debate got me thinking again about something that's been bothering me recently. I've been having a hard time getting my thoughts straight, and that's still true. I'll quote myself and then try to clarify:

A really good motto for a scientist who wants to be open could be this: "Be as open as I personally want to be." This is very different than "be as open as possible." What I am specifically thinking is that young scientists (i.e., not yet beaten-down) seem to usually have very natural tendencies towards open science. But the overall level of natural talent for openness may vary enough that "open notebook science" may just not be the best method of openness for some people. But everyone can strive to "be as open as they want to be", and resist pressure to be closed coming from outside (fear of scooping; lack of technical means; resistance from colleagues). In contrast to these external pressures, I think it may be legitimate for someone to want to be open, but also maintain some privacy so they can get a personal reward of doing something all by themselves, for example. Perhaps posting all of their electronic notes 6 months or a year down the line.

"Be as open as I want to be." I don't know if that has value for anyone else, but it a very powerful mission statement for me right now. It's powerful, because I really believe in it, but I am not achieving it. I'll talk about that later in the post. But, first I want to talk about it in a more positive light.

What kind of openness should be required?

I am starting to decide that I'm not going to try to force my lab members to do specific kinds of open science. I am thinking instead that my goal will be to remove as many barriers as possible so that my lab members can achieve the level of openness they desire. I believe that adults have unchangeable natural talents, and I think that scientists will be cutout for different kinds of openness. For example, Anthony in our lab has recently started doing open notebook science, true to its definition. I am really excited about this. He is a natural for ONS. I don't think that he has any problem writing anything in public. In fact, I think his notebook being open is a motivator for him to make it even better than he would a private notebook. This is the way he's wired, and it's not surprising if you know him. In contrast, I think some people would find that their creativity and drive are seriously hampered by doing ONS. For example, me as a graduate student. I don't know whether doing ONS would have worked or not. I actually kept what I think is a very good electronic lab notebook. But it was private, and I don't know whether I would have taken as many notes (and dropped as many F-bombs) if I knew it was public. I also don't know if I would have reacted well to someone posting a suggestion to me when I was immersed in trying to figure out something by myself. I do know that I would have been fine posting my notebook in public with some time delay. In fact, if anyone posts a comment to this blog asking me to post my grad school notebook in public, I'll go ahead and do that...f-bombs and all.

So, while I don't think I'll require ONS for all lab members, I may have other requirements, such as delayed notebook publishing. What I am worried about is hampering creativity and productivity of young scientists by striving for inappropriately selected open science goals. I do want my students (and postdocs in the future) to strive for open science, but I want them to do it in the way that best leverages their talents.

I am failing at my own principles

"Be as open as I want to be." I and our lab have made some great strides in the past few months towards this principle. For me, I think the transformation was fueled by a strong belief in the power and even morality of open science. But it did take a heavy dose of "what the fuck" to spark the flurry of steps I took this past winter break. (I think that may be my first f-bomb while blogging; I feel alive.) I'm happy and excited about what we're doing. But I'm also not achieving openness as much as I'd like. And I'm confused. Two themes dominate my struggles with openness:

• The students in my labs and their scientific careers
• My collaborators, their careers, and my gratitude for their assistance and mentoring
  

I'm not trying to sound altruistic here. One of my talents is that I get genuine happiness out of feeling like I've helped other people succeed. You can see that both of those items above feed that desire in me. I do think those two items are what is confusing me. In contrast, the issue of being scooped, in itself does not impact my thinking. I do worry about being scooped, but I have already concluded that being open does not increase the chances of being scooped. I believe being open decreases the chances of being accidentally scooped substantially. Furthermore, I even believe that being completely open would reduce the chances of being purposefully scooped. This is because the published track record would make it easier to shame the person who did the scooping.

Being scooped would be emotionally devastating. This is true. And it would have an impact on my lab and my students. This is what my students and I have been discussing the past couple years, and I think we've developed a collective (perhaps unspoken) understanding that we'll be OK even if that does happen. I think I can protect and rescue my students from that scenario. The collaborator issue is so much more complicated.

The collaborator issue is what is bothering me quite a bit now, and I really don't have any answer. Most of the scientists I know personally are "traditional." The ones I am trying to collaborate with are outstanding and highly respected by everyone, including me. The ones I am thinking about right now have put in a huge amount of effort helping me throughout various stages of my career. These traditional scientists, of course, are not Scientists 2.0, but they are fantastic scientists. I suspect, and in some cases directly know, that they would not approve of my science openness. So, I don't know how to deal with this external pressure towards closed science. The "what the fuck" strategy seems so disrespectful to people who've put energy into my career. But the "try to convince" them strategy is futile. "Showing them the way" will work...but at the risk of looking like "what the fuck" along the way and angering them. If we do get scooped, my students and I will be OK. But our mentors may never forgive us?

OK, I'm going to stop now...those are the challenges that are really bothering me this weekend.

Update on Our New Open Science Activities

I think it's been about a month since I started these blogs and joined friendfeed. It's been a whirlwind, really. I've e-met dozens of scientists around the globe in that short time, most of them much farther along than I and my lab are in terms of open science. The community of scientists out "here" is incredibly welcoming and helpful, and I want to send a thank you out to whomever of you read this post.

It's also been a time of huge change in terms of our lab's open science activities. We have started a lot of new open activities, and I thought I would make a list of them here. I'll only list those things that are new since mid-December, and I think it's quite a lot.

Blogging

Of course, I started blogging. We also started a blog that I and our lab members to contribute to. So far, only Anthony and I have contributed to it, but that will evolve over the year, I think. So far for me, blogging is a treat and I've been able to rationalize doing it based on potential synergies with the activities I'm supposed to be doing :)

Posting grants in public

We started posting our grants on Scribd. I chose this site from advice from Jean-Claude Bradley and Cameron Neylon. So far, I've liked the site as a place for sharing grants and other documents and it seems to work well. As an example, here is the grant we submitted last week. Posting grants has been really helpful so far and I expect it to continue to be helpful. We've received helpful comments from a couple people, and also made some science connections because of it. For example, Cameron and I realized we have a lot of science interests in common!

Paper preprint

A big step we took is we drafted our first paper out of our lab and we posted it on Nature Precedings. Larry Herskowitz is the lead author on this paper. We posted it a week ago, and we immediately received very helpful comments, questions, and suggestions from Richard Yeh. We're using OpenWetWare to talk about the paper with Richard and any others who want to join. My opinion right now is that OpenWetWare is a better place for these kinds of detailed lists of questions and suggestions, because we'll easily be able to break it into different topics, create sub-pages, and post supporting data, figures, etc. (In contrast, we found that trying to write the paper on a wiki just did not work for us at all.)

Open research projects

We have also taken some big steps towards "open notebook science" in the past month. We've been using a private wiki for about two years now, hosted by OpenWetWare. As I understand it, providing us with a private wiki was part of an experiment to see if it could draw in more users and lead them towards open science. You can't scientifically extrapolate from our experience, but then again, you don't have to approach it scientifically...so, my opinion is that providing the private wiki worked out beautifully for OWW's mission. I think they should continue to provide private wikis, including for select new users on a trial basis. It's quite possible (impossible to prove, though) that none of the open science activities I'm describing in this post would have been started had not Jason Kelly offered me the private wiki two years ago. Thank you Jason & all the OWW founders! I'd also like to thank Bill Flanagan who has helped me tremendously in many areas of the public and private wikis.

Having prepared via our "warmup time" with the private wiki, many of the students in my lab have begun to take open notebook steps in the past few weeks. You can find links to these on our open-research projects section. Anthony Salvagno has started doing real open-notebook science, keeping his daily notes on OWW, using the Lab Notebook system that Ricardo and others developed. Anthony is about to start learning molecular biology in our collaborator's lab, with guidance from Kelly Trujillo. The lab is not accustomed to e-notebooks, so it's going to be really tough for Anthony to not be driven to use a paper notebook. We'll see how it goes, I'm hoping he can show them the way!

Caleb Morse is embarking on some MediaWiki projects and we're trying to do our communication via OWW. There are many interesting things he might pursue this semester, many of them improvements to OWW and / or MediaWiki that can make the conduction of open research much easier. For example, he's currently working on modifying an extension to MW that uses cookies to prevent data loss when the browser crashes or closes while editing a page. This would be a huge plus for OWW.

Finally, Andy Maloney joined our lab in October and has learned to use the wiki very quickly. He recently took his first leap into the public wiki by posting his incredible instructions on how to build a laser diode control system from OEM parts. I'm also pushing him to post some of his earlier research accomplishments on OWW, including a custom microscope he built for imaging ultrasonic fields via the sonoluminescence. His Google SketchUp drawings and fly-by animations of the thing are amazing and I want you all to see them!

Open teaching

I've also started posting teaching material on Scribd. I'm trying to be careful about copyrighted material, so I'm not sure whether I can keep that up. One of the things I try to do after lecturing is to "debrief" to help with next year's lecture. So, combining blogging with Scribd is a good way to do that.

Back Pat

Looking back over that list of new things we started doing only recently has made me feel great about our lab. Obviously that rate of science "opening" can't continue. But I really do think we'll be able to keep up most of the things we've started, and I'm excited about that. On our private wiki, use a template that Anthony wrote for giving yourself a pat on the back. You just add {{BP}} to a page to use the template, and then you get an electronic pat on the back and feel good about yourself. Or at least some of use do. So, I'm going to put {{BP}} on this article and it's for me and all the students in our lab for these accomplishments. It's also a {{BP}} for all of the scientists I've been talking to recently and who have helped us take all these steps. They've provided very valuable advice and examples about how to do it, as well as encouragement and feedback for the steps we've taken. Thank you!