Forum: Data sharing via Bittorrent is coming to Biostar
15
gravatar for Istvan Albert
3.2 years ago by
Istvan Albert ♦♦ 73k
University Park, USA
Istvan Albert ♦♦ 73k wrote:

Hello Everyone,

We are adding bittorrent data sharing to Biostars.  Help us identify bugs and issues by creating a few torrents and adding them to posts on the test site. Also feel free to comment and provide suggestions and feedback. The description of how it works is at:

http://test.biostars.org/info/data/

An example post with data can be seen at:

http://test.biostars.org/p/101/

A few details on how it works:

  1. Torrents can get attached to posts, answers or comments
  2. A post may have multiple torrents attached.
  3. Biostars will attempt to connect the IP number of the Bittorrent peer connection to the IP number of the Biostar user account. This allows you to see who the person that shares the data is.
  4. Anonymous users cannot create torrents but they may share existing datasets. 
  5. Data may be shared without making it visible on Biostar (although this should not be considered a secure way to share data)

 (note: the test site will not log you into your old account since the emails are protected so don't report that as an issue)

biostar forum bittorrent • 2.3k views
ADD COMMENTlink modified 3.1 years ago by Roman Valls Guimerà480 • written 3.2 years ago by Istvan Albert ♦♦ 73k

Istvan, I guess you got notified now but visiting the following link:

http://test.biostars.org/info/data/

I get a 500, Internal Server Error, right now. Same in production.

ADD REPLYlink modified 3.1 years ago • written 3.1 years ago by Roman Valls Guimerà480

This is a test server and as such it may run different versions. We were troubleshooting an email handler for the Bioconductor folks so the data server was taken offline and the current version is being run.

ADD REPLYlink written 3.1 years ago by Istvan Albert ♦♦ 73k
5
gravatar for Dan Gaston
3.2 years ago by
Dan Gaston6.8k
Canada
Dan Gaston6.8k wrote:

Looks great. Two thoughts:

 

1) I think it would have been great to simply provide integration and support for BioTorrents( http://www.biotorrents.net/browse.php). Disclosure: I know and work with the author of BioTorrents, but I think it is a great tool that is currently seriously underutilised. 

 

2) One of the reasons Torrents are currently underutlisied in biology is that for many of is, it simply isn't an option. Institutional IT policies often straight out block torrent traffic and the IT admins have no desire to whitelist torrents with trackers hosted at specific places for legitimate purposes. They would rather just not deal with torrent traffic period.

ADD COMMENTlink written 3.2 years ago by Dan Gaston6.8k
2

Yes, I do expect that some institutions will block that traffic because it has a bad reputation. I don't know how prevalent that is. It will probably vary even within organizations, time will allow us to get a sense of that. Bittorrent is the simplest and most effective way to share large datasets and we have to start using it for legitimate reasons to erase the stigma.

If we can provide the level of utility that makes scientists make their needs clear their IT will adapt to that. 

As for integrating with other sites sadly I haven't heard of it BioTorrents so never reached out to them (though I have know of AcademicTorrents) Looks like BioTorrents has been published in PLoS CompBio: http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0010071

I am open to any kind of collaboration though.

 

ADD REPLYlink modified 3.2 years ago • written 3.2 years ago by Istvan Albert ♦♦ 73k

Yes BioTorrents was published. Morgan Langille is now a new faculty member here at the same University I am at and we have worked together in the past. I would be happy to point him to this and see if he is interested. I am not sure how much he still actively works with BioTorrents. Our University at least monitors for bittorrent traffic on the network and getting caught results in a loss of network privileges. 

ADD REPLYlink written 3.2 years ago by Dan Gaston6.8k

IT infrastructure and network problems can be solved easily when there is a will. The need to share data is on the rise and academic IT has to react soon. I doubt there is now a better solution to it than this one.

Banning whole bittorent protocol because of piracy is like banning http because of porn.

ADD REPLYlink modified 3.2 years ago • written 3.2 years ago by martenson380
2
gravatar for Roman Valls Guimerà
3.1 years ago by
Stockholm
Roman Valls Guimerà480 wrote:

I just heard at a conference that a big institution such as Max Planck blocks bittorrent traffic. I'm not sure if this has been brainstormed before, but I just had two features on that line that might be interesting to explore:

 

1) Provide an optional VPN service to BioStar users (crowdfunded?) and/or refer to third party services to circumvent institutional network restrictions.

2) Have a google maps overlay on the institutions that block legit bittorrent traffic.

 

Thoughts?

ADD COMMENTlink written 3.1 years ago by Roman Valls Guimerà480
1

It is definitely a problem with no easy solution at hand - you know maybe the answer is a new simple http based protocol that works like BitTorrent and we come up and define it. Thing is, it is not all that complicated.

ADD REPLYlink written 3.1 years ago by Istvan Albert ♦♦ 73k

I'm not sure that would be a good strategy. You'll have to implement support on clients, getting it accepted upstream, known by the userbase.... we have all this already in place today with bittorrent clients.

I'm not saying that VPNs would be that much easier for users to adopt, but definitely less effort when it comes to implementation and adoption.

ADD REPLYlink modified 3.1 years ago • written 3.1 years ago by Roman Valls Guimerà480

Hmmm...: http://peerjs.com/

 

Apparently it does require proxies anyway for NAT'd environments:

 

Are there any caveats?

A small percentage of users are behind symmetric NATs. When two symmetric NAT users try to connect to each other, NAT traversal is impossible and no connection can be made. A workaround is to proxy through the connection through a TURN server. The PeerServer cloud service does not provide a TURN server. You'll have to find your own. You can pass a TURN server into the Peer object options. This will allow your PeerJS app to work seamlessly for this situation

ADD REPLYlink modified 3.1 years ago • written 3.1 years ago by Roman Valls Guimerà480

http://webp2p.org/

ADD REPLYlink written 3.1 years ago by Roman Valls Guimerà480

This topic is getting more and more interesting:

https://github.com/feross/webtorrent

ADD REPLYlink written 3.0 years ago by Roman Valls Guimerà480
1
gravatar for Daniel
3.2 years ago by
Daniel3.5k
Cardiff University
Daniel3.5k wrote:

I think this is a great idea. I do think that some institutes might block torrent data outright though.

Does anyone know the fidelity of torrents? I know that it should be self checking but is that 100% accurate?

ADD COMMENTlink written 3.2 years ago by Daniel3.5k

each piece is protected by a cryptographic hash, altering that is practically unfeasible

ADD REPLYlink written 3.2 years ago by Istvan Albert ♦♦ 73k
1
gravatar for martenson
3.2 years ago by
martenson380
PSU
martenson380 wrote:

Seems working, however only 1 of 3 test torrents will download (the largest one).

see: https://www.dropbox.com/s/knozqs9gitmf6ai/Screenshot%202014-07-15%2016.17.25.png

ADD COMMENTlink written 3.2 years ago by martenson380

Strange may be some sort of traffic filtering or DHCP problem yesterday they worked but today the don't - both of those were shared via wireless computers at PSU.

The large file was shared from my home system.

ADD REPLYlink written 3.2 years ago by Istvan Albert ♦♦ 73k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 925 users visited in the last hour