As promised, my slides from the 2012 “Trends from the Trenches” talk at this year’s Bio-IT World Expo have been posted online.

The slideshare.net embed should be below; if that does not work please use this URL:
http://www.slideshare.net/chrisdag/2012-trends-from-the-trenches

Talk to your experiments (and the cloud!)

BioTeam has had this cooking in the lab for a while now but now we can discuss it publicly! We’ve been working with our partners Accelrys and BT to demonstrate what we think is a really cool concept – using Apple Siri on iOS devices to interact directly with a cloud platform (BT Compute) and scientific analysis software (Accelrys Pipeline Pilot).

The screencast above shows a real (not faked) example of the prototype system we’ve been working on. For the screencast we’ve merged two separate screens — the iOS iPhone screen as well as the Pipeline Pilot console so people can see what is happening on both sides.

Drop us a line at info@bioteam.net, visit our booth at BioITWorld’12 or leave a comment if you like.

Register here for the MiniLIMS Webinar.

If you can’t make it to the webinar today, the recorded event will be available from CHI.

Aaron K.

The entire presentation is available Under this link.

My slides are available as a PDF:  2012 Aspera Webinar

PRESS RELEASE

BioTeam Launches MiniLIMS^TM for Ion Torrent PGM Sequencer

Marco Island, FL, February 15, 2012 – BioTeam Inc. announces the availability of BioTeam MiniLIMS for Ion Torrent Personal Genome Machine (PGM). MiniLIMS is a powerful Web-based Laboratory Information Management System (LIMS) built for scientists who want a cost-effective solution that is fast, simple and flexible.  MiniLIMS will be available for direct purchase through Life Technologies WebStore after March 15, 2012.

MiniLIMS is the first independent software solution to connect directly to the Ion Torrent Server. Tight integration permits automated data transfer, speeds data processing, and eliminates many sources of error. MiniLIMS captures NGS workflow processes including sample receipt, sample prep, planned experiments, and results viewing. This agreement expands the Life Technologies-BioTeam collaboration. BioTeam provides extended service and support for the Ion Torrent Server and Suite Software.

“The BioTeam has demonstrated market-leading knowledge of the Torrent Suite Software, that drives primary analysis of semiconductor sequencing, through their extensive work on both the Ion Torrent Grand Challenges and Torrent Server support services.  Their practical working knowledge of the underpinnings of the Ion Torrent software makes them a strong choice to consider when choosing a LIMS partner, “said Mike Lelivelt, Director of Bioinformatics and Software Products, Ion Torrent

“Effective data management is an ongoing challenge for NGS users with many still using spreadsheets. MiniLIMS helps PGM users quickly gain improved control of their NGS data and derive more value from their NGS-based activities,” said Stan Gloss, Managing Director, BioTeam.

Among MiniLIMS’ core capabilities are:

• Simplified Data Entry. Customize data entry and display – without programming – to exactly match your samples, reagents, workflows, and instruments. Data stored in spreadsheets can be quickly and easily uploaded.

• Automatic Loading of PGM Data. Data is imported from the PGM into MiniLIMS automatically, hyperlinking all the sample, run, and preparation details. Upload your own files, putting all the data you need in one place.

• Powerful Report Generation. Create custom reports to see the state of your experiments at any time; there are pages that show the history of all data in the system.

The new MiniLIMS Ion Torrent plugin extends MiniLIMS to work with PGM data. Round trip data management allows scientists to plan experiments from MiniLIMS and see results from the PGM load automatically. MiniLIMS can alert users when data appears or pages change. Routine experiments can be configured using templates so a new run can be setup with a few keystrokes. Results can be delivered to users using the same project and sample overview pages that support Bioanalyzer, Nanodrop, or other lab processes. A graphical overview provides a snapshot of recent runs and analyses.

To see the most recent screenshots of MiniLIMS for PGM visit:

http://biote.am/73

About BioTeam

BioTeam is a high performance consultancy, serving the needs of the life sciences community since 2002. Our expertise includes cluster computing, large scale data storage, cloud computing, and semantic LIMS development. For more information, please visit http://bioteam.net

A case study in transferring a life sciences IT infrastructure fully into the cloud

“Over the past few years, many life science organizations have dabbled in cloud computing and explored infrastructure-as-a-service, with varying degrees of enthusiasm and commitment. But one Connecticut company has decided to go for broke—transferring its entire IT infrastructure onto the Amazon cloud.   “

If you’ve been following my previous posts about the PHP API (starting here) you know a little bit about how MiniLIMS is exchanging data with the Torrent Server via the nicely designed RESTful JSON interface. What I’d like to show you here is how that data exchange is materializing in the MiniLIMS plugin.

First, I want to show you really quickly how you install the Ion Torrent plugin (assuming you have MiniLIMS installed at /var/www/minilims):

$ cd /var/www/minilims/plugins
$ sudo cp ~/Downloads/IonTorrent.tar.gz .
$ tar xzvf IonTorrent.tar.gz

That’s it. Using the runtime integration capabilities of PHP, the Ion Torrent plugin will be loaded the next time you access a MiniLIMS page.

Our 1.0 plugin release isn’t particularly glitzy, but it takes advantage of the Torrent Server API to provide some of the key things that our early users have been interested in: planned experiment posting; run and report retrieval; instrument status; and file and quality results delivery.

Planned experiments are a nice facility provided by the Torrent Server that allows you to define a PGM run outside of the tiny console on the instrument itself. By setting them up in MiniLIMS and posting to the Torrent Server, you can maintain data integrity by selecting predefined Sample and Project records.

A PGM Planned Experiment in MiniLIMS

Once saved, the Planned Experiment can be posted to the Torrent Server and, therefore, selected from the console.

As most lab scientists will attest to, the same basic experiment is run repeatedly with just a different sample and/or project. PGM Planned Experiments can be saved as PGM Experiment Templates that can be reused for common experiment types. When creating a new planned experiment, just chose a template from the available list and your form will be populated with it’s values.

Like the Illumina and Roche 454 MiniLIMS plugins, the Ion Torrent offering features a run loader. The Ion Torrent run loader retrieves Experiments (runs), Results (reports), Quality Metrics and Lib Metrics from the Torrent Server. As described in the previous postings, the data types closely follow the API arrangement. Data can either be retrieved via the regular cron job or by using the “Refresh data” button on the Torrent Server page.

Torrent Server analysis result with link back to the Torrent Browser

MiniLIMS also provides lab personnel with a quick overview of the state of the instrument run and analysis based on the data from the most recent run. FTP status of the run and status of the currently running report (if there is one) is shown along with a list of the most recent runs and results.

Instrument overview showing status and recent results

End user results delivery is also made simpler by MiniLIMS. MiniLIMS projects, typically used to group user Samples, now include a summary of the results associated with your project. The summary has just about everything a discerning scientist would need: links to FASTQ and SFF files (zipped versions), drill-down links to full PGM Experiment and PGM Result pages, the full analysis reports on the Torrent Server and q17 and q20 read numbers.

MiniLIMS Project with PGM results table. More file links and q17/q20 results are available off page.

Remember when looking at these examples of the PGM plugin pages that MiniLIMS is end-user configurable. Though the data types used to supply a table result like the one above are fixed, specific columns can be added or removed, filters and sorting can be applied and new properties can be added to the Project definition to match your lab workflow.

Those are pretty much the key elements of the 1.0 plugin. We expect to have a couple more releases this year as MiniLIMS grows and we incorporate feedback from customers. If you’re going to AGBT, come find us.

In 2012 BioTeam will be offering two different classes – our popular “Fundamentals: Amazon Web Services for Science & Engineering” class along with a new 1-day course on practical systems and cloud orchestration that we are calling “Cloud Orchestration with Opscode Chef ”

Classes, dates and locations are as follows:

• (CANCELLED) Friday March 9, 2012:  Cloud Orchestration (in conjunction with XGen Conference & Expo in San Diego, CA USA)
• Sunday & Monday April 22-23, 2012 – AWS for Science & Engineering (in conjunction with BioITWorld in Boston, MA USA)
• April 27, 2012, Friday – Cloud Orchestration (in conjunction with BioITWorld in Boston, MA USA)
• May 18-19, 2012, Friday & Saturday – AWS for Science & Engineering (in conjunction with Biomarker World Congress, Philadelphia, PA USA)
• May 20, 2012, Sunday – Cloud Orchestration (in conjunction with Biomarker World Congress, Philadelphia, PA USA)
• June 14-15, 2012 – Thursday – Friday AWS for Science & Engineering in San Francisco, CA
• September 8-9, 2012 Saturday & Sunday AWS for Science & Engineering (in conjunction with MMTC Cloud Conference , San Francisco CA)
• September 14, 2012 – Friday – Cloud Orchestration (in conjunction with MMTC Cloud Conference , San Francisco CA)

Registration & More Information

http://www.healthtech.com/cloud

Our friends at Cambridge Healthtech Institute do all the hard work with training locations, logistics, advertising, enrollment and management. CHI should be considered the authoritative source for all training information and course details. For general questions not related to a specific scheduled class, feel free to contact chris@bioteam.net.

CHI’s primary page for the Amazon training class is http://www.healthtech.com/cloud and we’ll update this page if there is a separate course and enrollment page set up for the Chef Orchestration training.

• Ion Torrent Server API, I’d like to introduce you to the MiniLIMS API
• The MiniLIMS 4 column table and it’s PHP manifestation, the TypeInstance
• A quick round of applause for the Ion Torrent Server API

Bioinformatics has been notorious for the tension between public, supported access to data and algorithms and the often critical need to get under the hood. The ABI trace file and the Illumina output file system are good examples of data sources in which detailed internal access was necessary, but effective APIs did not exist and controlled, supported access was insufficient.

My approach to the data object portion of the API described below is to try to do some of both. I think I’ve found an accommodation of both structured access to the Torrent Server data objects and availability of the rawest form of data. If you know what you’re doing and are willing to take responsibility, take the raw data and go nuts. Otherwise, stick to the more managed parts of the API.

The Torrent Server RESTful API returns a JSON document that can be easily converted to a stdclass object via json_decode. If you’re not familiar with the stdclass, don’t feel bad; the only real documentation comes from third parties. It’s easy to understand, though; it’s what you get when you turn an associative array into an object. We’ll use this as the basis for the PHP API.

Here is an example of the JSON document returned by the previously described Torrent Server API, specifically for an instrument (“Rig” in Ion Torrent parlance):

JSON returned for an Instrument query

    $rigresult = json_decode($jsondoc);
        ...
    $rigresult->meta->limit == 20;
        ...
    $defaultrig = $rigresult->objects[0];
    $defaultrig->name == "default";

Easy, eh? We could probably just take these stdclass objects and convert them to MiniLIMS TypeInstances and we’d be done before lunch.

Yeah, you’re right. It’s a little too easy.

As I mentioned above, we’re going to try to provide some control and structure over the objects pulled from the server. This will not only help the general consumer of this PHP API, but it will also help the MiniLIMS system more clearly handle changes to the Torrent Server. Since you can’t add or change the methods of the stdclass, I’m going to use a wrapper object and call it PGMObject. From the PGMObject, more specific subclasses will be derived according to the Torrent Server types.

Before I describe the PGMObject, though, I’d like to use a subclass as an example of what I’d want to do. I’ll stick with the instrument data type since it’s relatively small.

PGM Instrument PHP class

There are 4 things I’m trying to accomplish here:

1. PGMInstrument extends PGMObject, the class that will handle all of the common functions
2. The properties array that is populated in the constructor lists the valid property names for this object so that we can check them.
3. The required array allows me to check for required fields in the validation method (coming up shortly)
4. The defaults array allows me to set defaults for a particular field.

This really is the essence of the distinction between one PGM object and another- a list of properties and the characteristics of those properties. How those properties and their characteristics are used can be defined in the base class for the most part by iterating through these arrays.

Below is a screenshot of the PGMObject interface (a collapsed object view from an Eclipse editor window).

PGMObject interface (collapsed object view in Eclipse)

First, I want to point out the constructor and the getJson/setJson methods. If you use the TorrentServer object (we’ll see that one soon) to retrieve a PGMObject from your Torrent Server instance, you can just grab the JSON document and do with it as you will. More often than not the object representations will be very handy, but, if you need low level access, it’s yours.

This JSON document represents a single one of the objects returned by the Torrent Server. A URL query typically returns an array of matching objects along with a {meta} document that can be used to support paging. The document maintained by these objects is a single data object with no associated meta.

If you don’t want to work with the raw JSON, you have two choices for interacting with the data values; either getPropertyValue/setPropertyValue or getters and setters for the individual properties.

The getPropertyValue/setPropertyValue method works just like the MiniLIMS methods of the same name; a property name and value are passed for setPropertyValue and just a property name for getPropertyValue. These are really handy for treating PGMObject subclasses generically and, particularly, for iterating through the full property list. If you wanted to write an HTML table you could do this:

$id  = "5";                           //An experiment id from the Torrent Server
$exp = $tserver->getExperiment($id);  //$tserver is a TorrentServer object

$html   = "<table><tr><th>Property</th><th>Value</th></tr>\n";
foreach($exp->getProperties() as $prop){
  $html .= "<tr>";
  $html .= "<td>" . $prop . "getPropertyValue($property) . "</td>";
  $html .= "</tr>\n";
}
$html  .= "</table>\n";

Property-specific getters and setters are nice to have for data objects like these. Popularized by Java, get/set methods for accessing object members allows for controlled access, but can be incredibly tedious to write for the most common case of simply getting or setting the attribute. The __call function is a special PHP object method that is called when an undefined function is called on an object. By having this function look for getters and setters that match the set of valid properties for the PGMObject, we can write one chunk of code to handle all of the routine get/set calls. Of course, if you want to do any special handling in a getter or setter (e.g. make something a computed value), you can define that function for your PGMObject subclass; if the method is explicitly defined, the __call function won’t get called.

Here is a chunk of __call where the parameter $name is the name of the function being called and $arguments is the array of argument names and values:

  public function __call($name,$arguments){
    if (!isset($this->properties)){
      throw new Exception("PGMObject cannot function without a properties list");
    }
    $matches = array();
    if (preg_match('/^get(.*)/',$name,$matches)){
      $prop = lcfirst($matches[1]);
      if (in_array($prop,$this->properties)){
        if (!isset($this->object)){
          if (isset($this->json)){
            $this->parseJson();
          }
          else {
            return "";
          }
        }
        return $this->object->$prop;
      }
    }
...
}

that allows you to do this:

$id  = "5";                           //An experiment id from the Torrent Server
$exp = $tserver->getExperiment($id);  //$tserver is a TorrentServer object

$chiptype   = $exp->getChipType();

without explicitly defining the getChipType() method.

The final “value-add” that we place on top of the PGMObject is the isValid() method. This is primarily to support objects that are being pushed to the Torrent Server (like a PlannedExperiment object). In the base class, this function goes through the $required array and makes sure that all of the properties listed have a value. If they do not, the method returns a null value and stores a message in the $validationErrors array.

This snippet shows how to use it:

$pexp = new PlannedExperiment();
$pexp->setId("5");
$pexp->setSample("Fish blood");
$pexp->setPlanPGM("default");
if (!$pexp->isValid()){
  print "Planned experiment is not valid for the following reason(s):<p/>\n";
  print implode("<p/>",$pexp->getValidationErrors());
}
else {
  $tserver->putPlannedExperiment($pexp);
}

Long-winded though this was, I hope the essential point is clear. I’ve tried to accommodate both raw(ish) access to the Torrent Server data objects and an object-oriented structure that can support controlled access, while at the same time minimizing the amount of routine coding.

In the next post, we’ll examine the TorrentServer class, whose purpose is to provide access to a Torrent Server, much like a traditional database object broker.

Excerpt:

In most organizations, the human resources of bioinformatics are a regular source of tension. Unless you’re particularly lucky, you can be plagued by politics, illogical decision making, disappointment, and low productivity. While you can have these problems in a properly-balanced organization, there are certain org charts in which they are endemic. 

The full article can be read at it’s online home:

http://www.bio-itworld.com/issues/2012/jan/unstable-equilibrium-bioinformatics-org-chart.html