Question: Making A Bioinformatics Application Available In The Amazon Ec2 Cloud
1
gravatar for Jlw
6.7 years ago by
Jlw180
USA/Storrs/UConn
Jlw180 wrote:

I have a user who is interested in having their application (custom c++/python application) hosted in the Amazon cloud and made accessible to the public. Configuration of the environment is sometimes difficult for users so this would make a VM immediately available to users. Use of the AWS Marketplace (via creation of an AMI) seems to be one method to do this. There is an option to host for free and allow unlimited access to the public for free (aside from the individuals own access to the cloud). There are some security regulations that lists limiting users ability to input their own data to the application (which in this case would be required). It also states this is limited to single CPU operations. Does anyone have experience hosting a non-web based application. What are the best approaches for academic software?

We are also open to looking outside of Amazon for the task if it would be better to use Google compute...or?

cloud • 3.6k views
ADD COMMENTlink modified 5.3 years ago by Giovanni M Dall'Olio27k • written 6.7 years ago by Jlw180

what about asking an academic lab to host your application ?

ADD REPLYlink written 6.7 years ago by Pierre Lindenbaum129k

Ah, it is already hosted on a website with download/installation instructions through an academic group. We are hoping to use this as another option for access

ADD REPLYlink written 6.7 years ago by Jlw180

This tutorial post might be useful: Introduction to AWS Cloud Computing

ADD REPLYlink written 5.3 years ago by Malachi Griffith18k
5
gravatar for Matt Shirley
6.7 years ago by
Matt Shirley9.3k
Cambridge, MA
Matt Shirley9.3k wrote:

Make it a web application. As long as the data are not too large, you can have the user upload a file, then have the server process run your application and return the result as a web page. If you don't have access to a web server in your lab, I would recommend DigitalOcean for the price and performance. Take a look at this example: http://tripod.mattshirley.com. It's something I created this year, and is by no means the best possible web application. Briefly it:

  1. Runs a Flask framework WSGI service that renders the site templates, and provides REST APIs
  2. When a user uploads data, the data are validated for consistency, checked for illegal characters
  3. If the data look safe and correct, the service sends a request to run the triPOD script to a Celery queue, which is limited to running X jobs at once
  4. The user is sent to a page that polls an API that returns success when the job is finished and results are ready
  5. Results are then retrieved using the Celery ID and rendered in a results template page

You could do something similar for your program, and since it's written partly in Python, Flask might also be a good framework for you to work with. An (outdated) version of the app I created is on Github: https://github.com/mdshw5/tripod-web.

ADD COMMENTlink modified 6.7 years ago • written 6.7 years ago by Matt Shirley9.3k

+1 for making a web app. However, it's not that easy to put it all together. I prefer ngninx + bottle.py stack. I've never used Celery. It seems really useful.

ADD REPLYlink written 6.7 years ago by Damian Kao15k

Thanks! That does look like a good option though there is no budget to pay for hosting (even though this cost is minimal) so it would have to be in an environment where users pay for time/space on the cloud but the application is available for free (my understanding of how marketplace can work).

ADD REPLYlink written 6.7 years ago by Jlw180
3
gravatar for Giovanni M Dall'Olio
5.3 years ago by
London, UK
Giovanni M Dall'Olio27k wrote:

You can make the app available as a docker container.

Advantages:

  • users will be able to run it on their computer or anywhere else, not only on amazon
  • It is still a virtual image, so users don't have to worry about configuration and installation of software
  • It should be easier to prepare than an amazon VM. You only have to write a text file.

Some links:

ADD COMMENTlink modified 5.3 years ago • written 5.3 years ago by Giovanni M Dall'Olio27k
2
gravatar for Istvan Albert
6.7 years ago by
Istvan Albert ♦♦ 84k
University Park, USA
Istvan Albert ♦♦ 84k wrote:

Your wording seems to imply that you want to allow access to the public to log in and use the program as a standalone application.

To which I say categorically no, you should not consider that option, your system will be hijacked and used for all kinds of other purposes.

ADD COMMENTlink written 6.7 years ago by Istvan Albert ♦♦ 84k

The idea is to host this on the Amazon cloud (not our own system) similar to how you would access other bioinformatics applications, such as an assembler or SNP caller. We are just trying to find the best medium to serve out this application.

ADD REPLYlink written 6.7 years ago by Jlw180
2

it does not matter where you host it, the idea of having a computer with free public access where users can log into a shell to run programs is not recommended by any means

ADD REPLYlink written 6.7 years ago by Istvan Albert ♦♦ 84k
0
gravatar for John
5.3 years ago by
John12k
Germany
John12k wrote:

Programs which were developed to be run by a trusted user cannot be easily ported over to the web, unfortunately. Anything that reads/writes to disk is going to be a security concern. Double or triple the risk ratio if the server-side code is also publicly available as part of an academic publication.

Of course, it can be done - but if you are not provided the budget even for hosting, i doubt you will be allowed to commit the resources required to maintain and secure a public-facing server.

Whats the worst that could happen? Well, in descending probability:
- The server will be used as a jumping off point for other more 'noisy' attacks, safe in the knowledge that if 100 totally unconnected people visit your site a day, tracking the actual attacker will be next to impossible.
- The website will be defaced by "p|-|ysici5ts_rul3" to contains images and text of how superior physics is as a natural science. or something...
- The website will start pushing "required java binaries" to users of the site.
- Ads appear everywhere.

I learnt every single one of these problems the hard way - and I am just supremely grateful that in all instances I noticed something was going on before someone from GCHQ came knocking on my door.

ADD COMMENTlink written 5.3 years ago by John12k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1007 users visited in the last hour