Question

Tutorial: Introduction to AWS Cloud Computing

44

3.9 years ago by

Washington University School of Medicine, St. Louis, USA

We have increasingly been using cloud computing environments to teach bioinformatics analysis skills. For example, we have been using AWS to conduct tutorials for some hands on bioinformatics workshops at CBW, NYGC, and CSHL. For many students learning bioinformatics in this way, "The Cloud" is a very abstract black box. When they do start to dig into the details, they encounter considerable jargon. Much of the existing documentation is written by system administration experts and is inaccessible to many biologists. We created the following tutorial in an effort to demystify some of this jargon and direct the reader to some of the documentation we found most useful. A table of contents for this tutorial is provided below to give you a sense of the topics covered. If there are other cloud computing tutorials that you have found useful, please link to them here.

Intro to AWS Cloud Computing

Cloud computing allows users to quickly access an arbitrary amount of compute resources from a distance without the need to buy or maintain hardware themselves. There are many cloud computing services. This tutorial describes the use of the Amazon Web Services (AWS) elastic compute (EC2) resource. However, the fundamental concepts covered here will generally apply to other cloud computing services such as Google Cloud, Digital Ocean, etc., though with substantial differences in jargon used by each provider.

It depends on the storage type where your data is. If it is on an ephemeral (Instance Store) volume you WILL lose your data when either stopping or terminating your instance. If your data is on an EBS volume it will persist until the EBS volume is deleted. Be aware that sometimes your EC2 instance might be set up to delete EBS volumes at termination (not usually when stopping) the instance. Otherwise it requires direct action to delete the EBS volume. You should read the sections: on Adding storage, Storage volume options, and What is difference between the start, stop, reboot and terminate instance states?.

ADD REPLY • link written 3.9 years ago by Obi Griffith ♦ 17k

If we install any tools, generally they go to /usr/bin or /usr/local/bin. Will the installed programmes be deleted once we stop the instance ? Should we install all the program in EBS only ? or should we setup the root storage as EBS ?

ADD REPLY • link modified 3.9 years ago • written 3.9 years ago by geek_y ♦ 9.3k

Yes, if the root volume is an ephemeral Instance Store volume you should not stop or terminate it unless you have already created a SnapShot or AMI (stored on EBS) to save the state. This issue can be avoided by using an EBS volume for the root storage when you create your instance. You can check the type of storage used for a root volume in your current instances in the EC2 console. More detailed discussion of these concepts is provided in the tutorial sections Obi indicated above.

ADD REPLY • link written 3.9 years ago by Malachi Griffith ♦ 17k

Have you considered shortening your presentation down to a 'Quick Start' guide?

ADD REPLY • link written 3.9 years ago by mcc • 80

1

That's a thought. We have been doing a very brief intro to cloud computing and then jumping right into analysis tasks at some workshops. However, we found that students find this very abstract and black box. This tutorial was created to really try to explain what is going on. If you just want to get rolling fast, you can log into AWS, hit the launch button and follow the steps in their wizard. You can mostly just pick the default options and you will be fine. We also wrote a tutorial that is much shorter but covers the same concepts for a publication that will be online soon: https://github.com/genome/gms/wiki/Beginners-Guide-to-installing-the-GMS-on-an-Amazon-Web-Services-%28AWS%29-Instance. All that being said, I think that if a beginner is going to work with AWS, they should do themselves a favor and devote a couple hours to understanding the basics. That investment will probably be saving you headaches pretty quickly...

ADD REPLY • link written 3.9 years ago by Malachi Griffith ♦ 17k

One more short question, Do you explain how to add resources to an instance? For example, I would like to grow my disk space and processing power by adding more cpus, etc.

BTW, Great job.

ADD REPLY • link written 3.9 years ago by mcc • 80

Good question. I believe for the most part, hardware configuration (CPUs, and memory) must be set when the instance is created and can not be adjusted. However, you can create an AMI of your instance, and then launch a new instance using that AMI. At that point you can choose an instance with more CPUs and memory. Similarly, if you want to expand the disk storage for an instance, you can create a snapshot of that instance, stop the instance and attach a larger one by following this procedure. If you just want to attach a new volume to a running instance, you can follow this procedure.

ADD REPLY • link written 3.9 years ago by Malachi Griffith ♦ 17k

Intro to AWS Cloud Computing

Table of Contents

Citation for this resource:

The following existing threads also seem relevant to this topic:

Similar posts • Search »