Overview
A leading academic medical center with an established on-premises HPC environment faced mounting challenges across reliability, stability, performance, and day-to-day operations. Researchers, computational scientists, and clinical research teams who depended on the cluster for biomedical and data-intensive scientific workloads were experiencing persistent disruptions, and the existing infrastructure could no longer keep pace with the demands of modern research computing.
BioTeam partnered with the organization to assess, redesign, and modernize the HPC environment from the ground up, redeploying existing hardware using state-of-the-art cluster and workload management approaches such as Warewulf and Slurm. The engagement combined remediation of the existing HPC environment, addition of new storage and GPU infrastructure, and a forward-looking operational maturity and scientific enablement strategy.
Impact Summary
- HPC cluster rebuilt from the ground up using Warewulf for cluster management and Slurm for workload scheduling
- Key cluster services deployed in a high-availability architecture, reducing single points of failure
- Rolling migration approach kept compute resources continuously available to active research programs throughout the transition
- New GPU nodes and high-performance research storage deployed as part of the modernization
- Cluster network upgraded from 40G to 100G/400G
- Open OnDemand deployed to provide researchers and scientific staff with easy-to-use, browser-based access to HPC resources
- HPC team trained and transitioned into ongoing operational ownership
- User-facing documentation and training developed to support researcher adoption and operational sustainability
Challenge
The client’s HPC environment had accumulated years of technical debt. Reliability issues were frequent, performance was inconsistent, and the operational model made it difficult for the HPC team to diagnose and resolve problems efficiently. Scientists and clinical research teams running computational workloads had limited confidence in the infrastructure, and unplanned downtime was impacting biomedical research productivity and scientific operations.
The cluster’s storage system had its own set of unresolved issues, and the network fabric was aging and undersized for current data volumes. The organization required more than incremental remediation efforts. What was needed was a comprehensive modernization strategy, executed without disrupting active scientific and research computing activities.
Approach
BioTeam began with a structured review of the existing infrastructure, including hardware, cluster management, job scheduling, scientific software, storage, networking, and the operational model used by the HPC team. This discovery process identified the root causes of instability and shaped the remediation strategy.
The cluster was redesigned from the ground up and redeployed on the existing hardware using a modern, DevOps-grounded approach. Core components include:
Cluster and Resource Management Warewulf was selected as the cluster provisioning and management system, with Slurm handling workload scheduling. Cluster services are deployed and managed through Ansible, enabling consistent, repeatable configuration management across the environment.
High Availability Architecture Key cluster services were deployed with high-availability failover configurations, eliminating the single-point-of-failure risks that had contributed to past reliability problems.
Rolling Migration Rather than taking the cluster offline during the transition, BioTeam executed the migration in rolling fashion. At every stage of the process, researchers and computational teams maintained access to critical compute resources supporting ongoing scientific work.
Infrastructure Expansion As part of the modernization, BioTeam led the design and deployment of new GPU nodes and high-performance research storage, directly addressing bottlenecks in the existing compute and I/O landscape. In collaboration with the client and vendors, the cluster network was upgraded from 40G to 100G/400G to support higher-throughput data movement and increasingly data-intensive biomedical workloads.
Scientific IT Services BioTeam designed and deployed Open OnDemand, giving researchers browser-based access to the HPC environment without requiring command-line expertise. This lowered barriers for less technical users and expanded access across multidisciplinary research teams.
Operations and Knowledge Transfer BioTeam provided day-to-day operational support for the modernized cluster while the client’s HPC team was trained on the new environment. Handoff was structured and deliberate, with user-facing documentation and formal training sessions supporting long-term operational ownership.
Outcomes
The client’s HPC infrastructure has been remediated and substantially modernized through a rolling migration approach, ensuring continuous compute availability for researchers throughout the process. The client’s HPC team is now equipped to manage the modernized environment independently.
Ongoing work focuses on continued operational maturity, refinement of scientific IT support services, and strategic planning for future infrastructure investments. BioTeam continues to provide scientific IT strategy and advisory support as the organization scales its research computing capabilities and supports increasingly complex biomedical research initiatives.
