Using Amazon Web Services?
3
15
Entering edit mode
11.1 years ago
Peter ▴ 170

Hello,

I am trying to use amazon web services for bioinformatics. I created a linux EC2 instance, but I've found that the space is limited to 10gB. I am attempting to take advantage of the EBS to create a larger hard drive space for me to work with. I am using the AWS console to attach the EBS to EC2 instance, but cannot figure out how to use the attached storage. Can anyone help me with the steps to make this all work?

Thanks!

cloud • 4.1k views
ADD COMMENT
1
Entering edit mode

10GB space sounds woefully inadequate to any modern computer. Are you sure you have it right? Check here.

ADD REPLY
0
Entering edit mode

This tutorial post might be useful: Introduction to AWS Cloud Computing

ADD REPLY
18
Entering edit mode
11.1 years ago
Alex ★ 1.5k

Useful EC2 tutorials for bioinformatics:

  1. Getting started guide: http://docs.amazonwebservices.com/AWSEC2/latest/GettingStartedGuide/
  2. Getting started JAVA: http://aws.amazon.com/articles/Amazon-EC2/3586
  3. Getting started Python: http://aws.amazon.com/articles/Amazon-EC2/3998
  4. EBS (this is MySQL example, but it is very useful and solves your current problems): http://aws.amazon.com/articles/Amazon-EC2/1663
  5. Hadoop tutorial: http://aws.amazon.com/articles/Amazon-EC2/873

To check available space try this:

[root@domU-12-31-39-00-B2-01 ~]# df -k
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda1             10321208   1636668   8160252  17% /
/dev/sda2            153899044    192072 145889348   1% /mnt

10Gb is the size of root disk, "other" disk space is in the /mnt directory.

For data analysis I usually use instance disk space ~1Tb. Note that all data will be deleted when you delete instance. And I save all important data to EBS (permanent storage). If you use US zone then network connection with NCBI is fantastic so expensive data keeping in permanent EBS is not necessary. Alternatively you can use public bioinformatics EBS, see full list here http://aws.amazon.com/datasets?_encoding=UTF8&jiveRedirect=1

Available biological public EBS:

  1. 1000 genomes
  2. Human Liver Cohort (Sage Bionetworks)
  3. C57BL/6J by C3H/HeJ Mouse Cross (Sage Bionetworks)
  4. Ensembl Annotated Human Genome Data - for FASTA
  5. Ensembl Annotated Human Genome Data - for MySQL
  6. Illumina - Jay Flatley (CEO of Illumina) Human Genome Data Set
  7. GenBank
  8. YRI Trio Dataset
  9. Ensembl - FASTA Database Files
  10. Unigene
  11. and more
ADD COMMENT
1
Entering edit mode

Just one suggestion: df -h would be more readable :)

ADD REPLY
0
Entering edit mode

Additionally, if you boot of an EBS-backed image, you can have up boot volumes up to 10 TB

ADD REPLY
0
Entering edit mode

Just realized I wrote 10 TB .. make that 1 TB

ADD REPLY
1
Entering edit mode
10.6 years ago
Marina Manrique ★ 1.3k

Maybe you just have to mount the EBS volume,

Here you can get very basics instructions to mount an attached volume to the instance.

Once you've attached the volume to the instance you just have to mount it with "mount" command

HTH

Marina

ADD COMMENT
1
Entering edit mode
10.6 years ago
Casbon ★ 3.2k

If you use an instance store based volume, rather than a EBS backed one, you will have ~300GB to play with at /mnt. However, it will not be persisted when the instance is deallocated.

Since you have 10GB, you are on EBS. This means you will need to attach a larger EBS at /dev/sdx and mount it as Marina points out.

ADD COMMENT

Login before adding your answer.

Traffic: 2368 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6