Is There Any Open And Free Computing Resources To Use Except Aws Free Tier To Run Hadoop Programm?
2
2
Entering edit mode
10.3 years ago
Fayue1015 ▴ 200

I have written a hadoop program which may need to run some large data set, I can use our local cluster which has 10 nodes, but it may take one month to finish, so I need to use some more nodes like 30-50 nodes, or better 100 nodes. so I can reduce the running time to several days. I roughly remember there are some free cloud resource to access for academic, but I forgot where are they. can anyone point me to some computing resources? Thanks a lot

• 6.0k views
0
Entering edit mode

Please indicate relevance of this question to a bioinformatics research problem.

3
Entering edit mode
10.3 years ago
Joachim ★ 2.9k

I am not aware of free cluster/cloud computing solutions that would permit you to use Hadoop. Perhaps you were thinking of Galaxy, but that is not what you are looking for.

Amazon Web-Services

You mentioned Amazon's free-tier, but I think you would quickly exceed the granted free computing time of 750h per month (on tiny micro instances -- mind you). However, it still might be financially viable to use Amazon's Web-Services, which come with Hadoop specific map-reduce configuration instructions; "spot instances" can be used to get extremely cheap EC2 Linux virtual machines up and running.

For example, last week I ran "High-Memory Quadruple Extra Large" instances (68GB memory, 8 cores, 1.6TB storage) at $US0.14 per hour, which normally cost$US1.80 per hour when rented as "on-demand instances." The only drawback of using spot-instances is that you are not guaranteed that they will run as long as you require, because you set a maximum price you are willing to pay and the hourly recalculated $US-price per hour might exceed your maximum price bid. Google Compute Engine You could also enquire with Google's Compute Engine. Google is currently giving preview access to selected members of the community. I did not get in, but perhaps you have better luck. Hope that helps. ADD COMMENT 0 Entering edit mode thanks for sharing! ADD REPLY 0 Entering edit mode btw, how do you know google compute engine select you or not, will they send you a email? I have registered and receive no confirmation ADD REPLY 0 Entering edit mode 7.4 years ago Please let me know, if the price still valid... We also have similar requirement. As I see the post is 2.9 years old. ADD COMMENT 0 Entering edit mode Microsoft Azure gives a free tier of 125$ valid for one month: https://azure.microsoft.com/

Google Genomics provides a free tier 300\$ for 60 days: https://cloud.google.com/genomics/#get-started