Intel hosted a seminar series event at the Broad Institute in Cambridge, MA on 9/6/2012. The topic was Managing Big Data in Life Sciences and Healthcare, a critically important topic in life sciences at the moment. I was invited to give a talk about how BioTeam sees the state of the art and how we approach the management of this expansive sea of data. Since we see an incredibly broad range of implementations of data management solutions, my presentation covered the 10,000ft overview of how I see the state of big data storage and the new challenges that are being faced. Since we often approach data management issues by implementing our own MiniLIMS software, I demonstrated how a system like MiniLIMS can help manage sprawling file systems from a research perspective.

My slides can be viewed below. If you can’t see the embedded presentation, please follow this link to SlideShare: http://www.slideshare.net/arieberman/ari-berman-intel-big-data-seminar-962012.

BioTeam’s Ari Berman authors this month’s Inside The Box Column:

http://www.bio-itworld.com/news/07/16/12/Inside-the-Box-Solutions-data-sharing-life-sciences.html 

Excerpt:

“Sharing scientific data is as fundamental to the progression of science as the research design itself. Without data sharing, experiments cannot be peer-reviewed, and scientists cannot perpetuate existing findings by taking the next steps in the laboratory.   

Unfortunately, data sharing is becoming more and more difficult. Compare scientific papers published in the early 90’s to those published in 2012 — the differences are striking. Back then, any and all data associated with a project could fit in a figure or two, so the paper itself was the point of data sharing. Today, more and more papers are published with reams of supplementary data, e.g. PDF tables can reach hundreds of pages and are themselves a distilled and reduced version of the original data. (New initiatives such as the journal GigaScience and its associated database should help address this issue.) This illustrates the crux of the issue: modern research produces tons of data and publications are no longer a viable medium for sharing all those data.”  

As promised, my slides from the 2012 “Trends from the Trenches” talk at this year’s Bio-IT World Expo have been posted online.

The slideshare.net embed should be below; if that does not work please use this URL:
http://www.slideshare.net/chrisdag/2012-trends-from-the-trenches

Talk to your experiments (and the cloud!)

BioTeam has had this cooking in the lab for a while now but now we can discuss it publicly! We’ve been working with our partners Accelrys and BT to demonstrate what we think is a really cool concept – using Apple Siri on iOS devices to interact directly with a cloud platform (BT Compute) and scientific analysis software (Accelrys Pipeline Pilot).

The screencast above shows a real (not faked) example of the prototype system we’ve been working on. For the screencast we’ve merged two separate screens — the iOS iPhone screen as well as the Pipeline Pilot console so people can see what is happening on both sides.

Drop us a line at info@bioteam.net, visit our booth at BioITWorld’12 or leave a comment if you like.

A case study in transferring a life sciences IT infrastructure fully into the cloud

“Over the past few years, many life science organizations have dabbled in cloud computing and explored infrastructure-as-a-service, with varying degrees of enthusiasm and commitment. But one Connecticut company has decided to go for broke—transferring its entire IT infrastructure onto the Amazon cloud.   “

Talk slides embedded below via slideshare.net – comments welcome on the use of slide share, this is sort of an experiment for us. You can download the full presentation file as well by following the link back to their site.

Backblaze Pod Project: 135 Terabytes (raw) for $12,000

This is a placeholder page to centralize links related to the Backblaze 2.0 storage project. As we add more articles, posts or resources we’ll update this “central” post page.

Twitter screwup; the link to the “Why you should never build a backblaze pod” post is here:
Why You Should Never Build A Backblaze Pod

Articles, Posts & Pictures

• Why You Should Never Build A Backblaze Pod

Why we built a backblaze

Real World Backblaze Costs

Backblaze Initial Performance Data

System assembly picture gallery

As promised, here is a PDF version of the slide presentation that Chris Dagdigian presented at the 2011 BioITWorld Expo conference. This is his “2011 HPC Trends from the Trenches” talk that crossed Track-1 (Infrastructure/Hardware), Track-2 (Infrastructure/Software) and Track-3 (Cloud).

Back in the day …

Way back in 2009 I placed an aging copy of my Grid Engine Administration training materials online. Response has been fantastic and it’s still one of the more popular links on this blog.

Today

Well it’s probably past time I did something similar aimed at people actually interested in using Grid Engine rather than just administering it.

It’s not comprehensive or all that interesting but I am going to post a bunch of slides cherry picked from the collection of things I build custom training presentations from. Think of them as covering an intro-level view of Grid Engine use and usage.

Intro to Grid Engine Usage & Simple Workflows

There are two slide decks, both of which are mostly stripped of information that is unique to a particular client, customer or Grid Engine cluster.

The first presentation is a short and dry introduction aimed at a user audience – it explains what Grid Engine is, what it does and what the expectations of the users are. It then dives into commands and examples.

The second presentation is also aimed at a basic user audience but talks a bit more about workflows, pipelines and simple SGE features that make life a bit easier for people who need to do more than a few simple ‘qsub’ actions.

Grid Engine Introduction for Users

Simple Workflows with Grid Engine

Hope they are useful!

I’m posting these online in the hopes that they may be useful to some people. Your comments and feedback are welcome. All I ask is that our effort to assemble this material be respected – you are welcome to download, re-distribute etc. but please don’t be lame and strip our company out of the materials or otherwise try to pass these files off as your own creation.