Question: Is There Any Open And Free Computing Resources To Use Except Aws Free Tier To Run Hadoop Programm?
2
gravatar for Fayue1015
8.5 years ago by
Fayue1015200
Singapore
Fayue1015200 wrote:

I have written a hadoop program which may need to run some large data set, I can use our local cluster which has 10 nodes, but it may take one month to finish, so I need to use some more nodes like 30-50 nodes, or better 100 nodes. so I can reduce the running time to several days. I roughly remember there are some free cloud resource to access for academic, but I forgot where are they. can anyone point me to some computing resources? Thanks a lot

• 4.9k views
ADD COMMENTlink modified 5.6 years ago by vakiti.sreekanth0 • written 8.5 years ago by Fayue1015200

Please indicate relevance of this question to a bioinformatics research problem.

ADD REPLYlink written 8.5 years ago by Neilfws49k
3
gravatar for Joachim
8.5 years ago by
Joachim2.9k
San Francisco, California
Joachim2.9k wrote:

I am not aware of free cluster/cloud computing solutions that would permit you to use Hadoop. Perhaps you were thinking of Galaxy, but that is not what you are looking for.

Amazon Web-Services

You mentioned Amazon's free-tier, but I think you would quickly exceed the granted free computing time of 750h per month (on tiny micro instances -- mind you). However, it still might be financially viable to use Amazon's Web-Services, which come with Hadoop specific map-reduce configuration instructions; "spot instances" can be used to get extremely cheap EC2 Linux virtual machines up and running.

For example, last week I ran "High-Memory Quadruple Extra Large" instances (68GB memory, 8 cores, 1.6TB storage) at $US0.14 per hour, which normally cost $US1.80 per hour when rented as "on-demand instances." The only drawback of using spot-instances is that you are not guaranteed that they will run as long as you require, because you set a maximum price you are willing to pay and the hourly recalculated $US-price per hour might exceed your maximum price bid.

Google Compute Engine

You could also enquire with Google's Compute Engine. Google is currently giving preview access to selected members of the community. I did not get in, but perhaps you have better luck.

Hope that helps.

ADD COMMENTlink modified 13 months ago by _r_am32k • written 8.5 years ago by Joachim2.9k

thanks for sharing!

ADD REPLYlink written 8.5 years ago by Fayue1015200

btw, how do you know google compute engine select you or not, will they send you a email? I have registered and receive no confirmation

ADD REPLYlink written 8.5 years ago by Fayue1015200
0
gravatar for vakiti.sreekanth
5.6 years ago by
vakiti.sreekanth0 wrote:

Please let me know, if the price still valid... We also have similar requirement. As I see the post is 2.9 years old.

ADD COMMENTlink modified 13 months ago by _r_am32k • written 5.6 years ago by vakiti.sreekanth0

Microsoft Azure gives a free tier of 125$ valid for one month: https://azure.microsoft.com/

Google Genomics provides a free tier 300$ for 60 days: https://cloud.google.com/genomics/#get-started

ADD REPLYlink modified 13 months ago by _r_am32k • written 5.6 years ago by Giovanni M Dall'Olio27k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1632 users visited in the last hour
_