BioTeam » storage

Intel Managing Big Data Event – Slides

ari — Wed, 19 Sep 2012 23:00:27 +0000

I’ve posted my slides from a Big Data event hosted by Intel on Sept 6th, 2012.

Intel hosted a seminar series event at the Broad Institute in Cambridge, MA on 9/6/2012. The topic was Managing Big Data in Life Sciences and Healthcare, a critically important topic in life sciences at the moment. I was invited to give a talk about how BioTeam sees the state of the art and how we approach the management of this expansive sea of data. Since we see an incredibly broad range of implementations of data management solutions, my presentation covered the 10,000ft overview of how I see the state of big data storage and the new challenges that are being faced. Since we often approach data management issues by implementing our own MiniLIMS software, I demonstrated how a system like MiniLIMS can help manage sprawling file systems from a research perspective.

My slides can be viewed below. If you can’t see the embedded presentation, please follow this link to SlideShare: http://www.slideshare.net/arieberman/ari-berman-intel-big-data-seminar-962012.

Ari Berman – Intel Big Data Seminar 9/6/2012 from arieberman

High Speed Data Transfer

cdwan — Tue, 03 Apr 2012 16:44:19 +0000

I recently had the opportunity to present a brief overview of some of the work that BioTeam has been doing with the New York Genome Center. The format was a “webinar,” sponsored by Aspera and hosted by Bio-IT World. I spoke briefly about the data motion needs of the center. The most interesting piece to me was an example where we achieved sustained data transfers of 700Mb/sec coast to coast over the commercial internet. At those data rates we do not need to rely on primitive hacks like fedexing hard disks. Michelle Munson, CEO of Aspera, spoke second. She explained how their software works as well as some of their plans for the future.

The entire presentation is available Under this link.

My slides are available as a PDF: 2012 Aspera Webinar

Panasas PAS 12 Unboxing

chrisdag — Tue, 01 Mar 2011 15:38:37 +0000

I’ve had a Pansasas PAS-12 in my home lab for longer than I’d care to admit, now that it’s almost time to box the system up and send it away I figured it would be good to post some pictures of the system (and the unboxing …)

What follows are some of my favorite pictures. The full set can be found in my Infrastructure Flickr stream …

One box for the director & storage blades, one box for the chassis itself

Packaged blade components

Unwrapped storage blade

Packaging is very well done, the layer underneath the bare chassis
contains power supplies, battery unit and the misc. cables & switching blades.

Battery unit macro shot

Macro shot of a switching blade 1

Switch blade macro 2

Switch blade macro 3

Blades partially installed

The system is boostrapped by connecting a serial cable to a director blade

Initial configuration via serial cable (and the very handy Keyspan USB-to-Serial adapter dongle!)

Blade macro shot

Almost online now …

Storage talk slides from BioMed-HPC Leadership Summit

chrisdag — Mon, 18 Oct 2010 15:32:05 +0000

As promised, here are the talk slides from the 2010 HPC Biomedical Leadership Summit where I was part of a morning talk session focusing on storage, large data and peta-scale data management issues. There were a number of interesting speakers but we were all limited to about 10 minutes duration so this slide deck is lighter than my usual stuff. To download my slides on “Grunt’s Eye View: Storage 2010″ please click on the image above or use this link: 2010-HarvardHPC_cdag_v2.pdf

Bio-IT World Europe 2010 – Science Centric Storage

chrisdag — Fri, 15 Oct 2010 13:25:50 +0000

Apologies for the delay in posting this PDF. For some reason I can’t shrink this down to a reasonable size, possibly because of the amount of pictures included in the original presentation. The PDF is about 19MB (sorry!) and I’ll try to shrink it later if I can. Click on the image above or follow this link to download it: 2010-BioITWorld-Europe_v6.pdf

Storage for Next-Gen Sequencing

blogadmin — Fri, 16 Oct 2009 19:14:12 +0000

Here are the talk slides from our 2009 Workshops (USA and Europe) on Storage Issues for Next-Generation Sequencing. The US Workshop was video taped, presentations will be online shortly.

Click on the links or the images below to download the presentations

Chris Dagdigian (BioTeam Inc): “Data & Data Management for Next-Generation Sequencing“
Jacob Farmer (Cambridge Computer Inc.): “Scalable Storage for Life Science“
Guy Coates (Wellcome Trust Sanger Institute): “New Sequencing Storage“
Matthew Trunnell (Broad Institute): “Issues in Data Storage and Data Management in Large-Scale Next-Gen Sequencing“

Big Storage

chrisdag — Wed, 17 Sep 2008 13:54:22 +0000

82 terabyte desktop folder, originally uploaded by ChrisDag.

This screencap is from my Apple laptop; it’s a NFS mount to a *really big* folder hosted on the network.

~90 terabytes of storage in a single namespace/folder costs a million dollars or more if you go with the Tier 1 vendors and have to pay for a backup solution.

The system backing this storage volume cost a few hundred grand and that includes all the misc. infrastructure to support a 72 core Linux cluster, racks, networking, UPS batteries, redundant fiber channel switches, multipath storage access via clustered fileservers and a large tape library robot for backups.

We really like this system, expect to see BioTeam talking about the configuration and purchase process in an upcoming conference or whitepaper.

Props to: Chris Dwan, Mike Cariaso from BioTeam and Sam Farmer from Cambridge Computer.