HUBzero on AWS – Stage 1

25 Dec 2012 HUBzero on AWS – Stage 1

hubzero-aws-003As mentioned on the http://bioteam.net/2012/12/hubzero-on-the-cloud/ “landing page” the process for using Opscode Chef to deploy a functional HUBzero platform onto Amazon requires four distinct stages.

The very first thing we need to do is take the “naked” Debian 6 server image we just launched and bootstrap the Opscode Chef client software onto it and then bind it to our configuration server. Bootstrapping chef onto the node turns the server into a fully managed and configurable target for all of our remaining configuration tasks.

There are many ways to bootstrap a node, our favorite is to just use the Opscode ‘knife’ command line utility. Knife knows how to launch EC2 server instances and if we feed it the proper parameters it also knows how to login to the fresh node and execute the commands needed to download, install and configure the chef-client system.

This is what our bootstrap script looks like, we are using a 3rd party bootstrap template that understands Debian 6 …

This knife command does nothing more than launch an official Debian server image using the Amazon parameters we provide for location and server size. It then waits until the SSHD daemon is responsive on the public IP address. Once the SSHD daemon is reachable; knife will SSH in and remotely execute all of the commands necessary to download and install Chef-client (along with the certificates and info needed to point the chef-client at the BioTeam configuration server).

#!/bin/sh

knife ec2 server create 
 --verbose 
 --template-file /Users/dag/opscode-platform-projects/bioteam/chef-repo/.chef/bootstrap/debian6-gems.erb 
 --availability-zone us-east-1b 
 --security-group dag-HZ 
 --node-name hubzeroTestNode 
 --flavor t1.micro 
 --image ami-4d20a724 
 --ssh-user admin 
 --identity-file /Users/dag/opscode-platform-projects/bioteam/bioteam-IAM-admins-v1.pem 
 --ssh-key  bioteam-IAM-admins-v1

Screencast: Bootstrapping the official Debian image

Note: if the text is unclear, use the larger viewer or switch to one of the HD-quality streaming formats.

Now that we have a debian box dancing to our tune …

Now it’s time to start installing HUBzero. In the next “phase” we are going to add a Chef “role” to this node with a “recipe” run list that looks like this:

run_list:
    recipe[motd]
    recipe[hostname::hubzero-standalone]
    recipe[users::hubzero]
    recipe[sudo]
    recipe[exim4-light]
    recipe[mysql::server]
    recipe[mysql::server_ec2]
    recipe[hubzero::dependencies]
    recipe[hubzero::sources]

Lets break down the individual chef recipes in this “role”:

  • recipe[motd] – This just adds a BioTeam login banner via uploading a new /etc/motd file
  • recipe[hostname::hubzero-standalone] – Change EC2 server hostname to use the full public hostname; adjust /etc/hosts as well
  • recipe[users::hubzero] – Create an alternate admin account (‘hubadmin’) and install the administrator SSH key
  • recipe[sudo] – Add the new admin ‘hubadmin’ user account to the /etc/sudoers list
  • recipe[exim4-light] – Install and configure the exim4 mail server (HUBzero likes to send email)
  • recipe[mysql::server] – Install and configure MySQL client and server
  • recipe[hubzero::dependencies] – Start installing some Debian OS software packages that we know HUBzero will need later
  • recipe[hubzero::sources] – Adjust /etc/apt/sources.list to add the HUBzero software repository URLs and repo signing key

The above recipes roughly correspond to Steps 1.0 – 1.3 in the HUBzero manual install guide located at http://hubzero.org/documentation/1.1.0/installation.

Why stop now? Why not continue with all the other software?

There are a few reasons to “stop” at this point …

  • We need to delete the ‘admin’ user that is configured as the EC2 admin account within this server image. The admin user account that comes with the Official Debian image conflicts with HUBzero requirements that all user accounts with UID greater or equal than 1000 must be managed by the Hub. Since the ‘admin’ account in this server image has UID 1000 it needs to be blown away.
  • As part of the hub zero::dependencies recipe we installed a new OpenVZ-compatible Linux kernel. We need a reboot to load the new kernel

So at this point it makes sense to stop here, reboot and then continue all future admin steps via the new ‘hubadmin’ admin account…

Screencast: Loading and running role[hubzero-standalone]

Note: if the text is unclear, use the larger viewer or switch to one of the HD-quality streaming formats.

Related Posts
Filter by
Post Page
Events Featured Employee Posts Tech Notes Screencasts
Sort by
2 Comments
  • Daniel Kohn
    Posted at 13:36h, 03 November Reply

    Hi Chris,

    Did you guys ever go beyond this point?
    We are exploring hubzero as a community-ware solution for artists and scientists collaborating in NY and have been looking at different hosting options. A programmer friend who uses EC2 pointed me your way.

    Thanks,

    Daniel.

    • bioteam
      Posted at 13:40h, 03 November Reply

      Daniel – check with the HubZero foundation. We worked with them for some time and their internal devs are now totally up to speed on AWS. I know they have very serious plans to roll out HubZero on Amazon but I’m not sure what info I have that could be considered confidential/early/private at this moment. Expect something sooner rather than later but rather than me re-posting gossip you should approach them directly to get the current status. Good luck! –Chris

Post A Comment