BioTeam is presenting two talks at the iRODS User Group Meeting 2017 taking place next week in Utrecht, The Netherlands. John Jacquay and Aaron Gardner will be sharing many of the software and architectural design patterns BioTeam has been using to incorporate iRODS into our development stack. For those attending feel free to introduce yourselves, we would be happy to meet you. The iRODS consortium will be making slides and video of the presentations available after the meeting concludes.
Full abstracts for BioTeam’s talks are shown below.
Building Data Systems with iRODS and Golang
John Jacquay and Aaron Gardner
iRODS is a powerful core component of data management systems. By harnessing the ideas of data cataloging, workflow automation, and abstraction, iRODS provides developers a comprehensive set of features that act as a framework for intricate, automated, data storage systems. However, the same attributes that give iRODS it’s power and flexibility, also add to its complexity and steep learning curve.
In today’s constantly evolving, rapid-pace software landscape, developers expect a certain level of accessibility and low barrier of entry when learning new technologies. Typically, new developers would not associate these attributes with iRODS or C++. This is because a paradigm shift has taken place in how computing, new ideas, methodologies, and architectures are being developed and shared. Golang and its surrounding community is one of the modern programming languages at the epicenter of this advancement. Usage of Golang in production deployments has been growing at an unprecedented pace, because the language fills the niche of being a simplistic, fast, scalable, and productive language to write code in. Golang’s simple, opinionated, and idiomatic design allows common patterns and principles to be shared between software of distinct domains.
By combining the benefits of Golang and its developer community with the extendable power of iRODS, developers may find themselves happier and more productive. This talk will introduce the audience to several efforts completed and underway that harness the power of iRODS with Golang, and will conclude with a demonstration of how easy it is to write efficient code when building iRODS based data systems using Golang.
A Converged, Fault Tolerant, Distributed Parallel Architecture for iRODS
Aaron Gardner, John Jacquay, Alex Oumantsev and Markus Dittrich
Traditionally, fault tolerant iRODS data grids have been constructed using dedicated physical or virtual servers for each service component, with data and service availability in the event of failure often maintained through replicas and active-passive service configurations. When replicating and distributing the iCAT, its backing relational database, iRES resource servers, and corresponding managed data it has been difficult to balance efficient use of storage media and server hardware while simultaneously achieving high availability and performance.
One of iRODS key strengths is its ability to flexibly fit into nearly any storage architecture, but that strength has become a weakness for organizations new to iRODS who are seeking a reference design that provides performance and high-availability within a minimal initial hardware footprint. This talk will present a new reference system level architecture that distributes and parallelizes the iRODS resource server workload across a system backed by a high-performance clustered filesystem. This architecture is able to maintain a coherent view of single replica data across iRODS resources with as little as a single iRODS resource server instance available. Furthermore, failover and failback of resource servers, iCAT servers, and the backing relational database has been automated. An example real-world deployment will be presented where the distributed parallel resource server and highly-available iCAT service configuration have been virtualized and deployed onto a single active-active storage controller pair along with an embedded clustered filesystem.
