Forum:Data sharing via Bittorrent is coming to Biostar
4
15
Entering edit mode
10.3 years ago

Hello Everyone,

We are adding bittorrent data sharing to Biostars. Help us identify bugs and issues by creating a few torrents and adding them to posts on the test site. Also feel free to comment and provide suggestions and feedback. The description of how it works is at:

http://test.biostars.org/info/data/

An example post with data can be seen at:

http://test.biostars.org/p/101/

A few details on how it works:

  1. Torrents can get attached to posts, answers or comments
  2. A post may have multiple torrents attached.
  3. Biostars will attempt to connect the IP number of the Bittorrent peer connection to the IP number of the Biostar user account. This allows you to see who the person that shares the data is.
  4. Anonymous users cannot create torrents but they may share existing datasets.
  5. Data may be shared without making it visible on Biostar (although this should not be considered a secure way to share data)

(note: the test site will not log you into your old account since the emails are protected so don't report that as an issue)

biostars bittorrent • 5.5k views
ADD COMMENT
0
Entering edit mode

Istvan, I guess you got notified now but visiting the following link:

http://test.biostars.org/info/data/

I get a 500, Internal Server Error, right now. Same in production.

ADD REPLY
0
Entering edit mode

This is a test server and as such it may run different versions. We were troubleshooting an email handler for the Bioconductor folks so the data server was taken offline and the current version is being run.

ADD REPLY
5
Entering edit mode
10.3 years ago
DG 7.3k

Looks great. Two thoughts:

  1. I think it would have been great to simply provide integration and support for BioTorrents. Disclosure: I know and work with the author of BioTorrents, but I think it is a great tool that is currently seriously underutilised.
  2. One of the reasons Torrents are currently underutlisied in biology is that for many of is, it simply isn't an option. Institutional IT policies often straight out block torrent traffic and the IT admins have no desire to whitelist torrents with trackers hosted at specific places for legitimate purposes. They would rather just not deal with torrent traffic period.
ADD COMMENT
2
Entering edit mode

Yes, I do expect that some institutions will block that traffic because it has a bad reputation. I don't know how prevalent that is. It will probably vary even within organizations, time will allow us to get a sense of that. Bittorrent is the simplest and most effective way to share large datasets and we have to start using it for legitimate reasons to erase the stigma.

If we can provide the level of utility that makes scientists make their needs clear their IT will adapt to that.

As for integrating with other sites sadly I haven't heard of it BioTorrents so never reached out to them (though I have know of AcademicTorrents) Looks like BioTorrents has been published in PLoS CompBio: http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0010071

I am open to any kind of collaboration though.

ADD REPLY
0
Entering edit mode

Yes BioTorrents was published. Morgan Langille is now a new faculty member here at the same University I am at and we have worked together in the past. I would be happy to point him to this and see if he is interested. I am not sure how much he still actively works with BioTorrents. Our University at least monitors for bittorrent traffic on the network and getting caught results in a loss of network privileges.

ADD REPLY
0
Entering edit mode

IT infrastructure and network problems can be solved easily when there is a will. The need to share data is on the rise and academic IT has to react soon. I doubt there is now a better solution to it than this one.

Banning whole bittorent protocol because of piracy is like banning http because of porn.

ADD REPLY
2
Entering edit mode
10.2 years ago

I just heard at a conference that a big institution such as Max Planck blocks bittorrent traffic. I'm not sure if this has been brainstormed before, but I just had two features on that line that might be interesting to explore:

  1. Provide an optional VPN service to BioStar users (crowdfunded?) and/or refer to third party services to circumvent institutional network restrictions.
  2. Have a google maps overlay on the institutions that block legit bittorrent traffic.

Thoughts?

ADD COMMENT
1
Entering edit mode

It is definitely a problem with no easy solution at hand - you know maybe the answer is a new simple http based protocol that works like BitTorrent and we come up and define it. Thing is, it is not all that complicated.

ADD REPLY
0
Entering edit mode

I'm not sure that would be a good strategy. You'll have to implement support on clients, getting it accepted upstream, known by the userbase.... we have all this already in place today with bittorrent clients.

I'm not saying that VPNs would be that much easier for users to adopt, but definitely less effort when it comes to implementation and adoption.

ADD REPLY
0
Entering edit mode

Hmmm...: http://peerjs.com/

Apparently it does require proxies anyway for NAT'd environments:

Are there any caveats?

A small percentage of users are behind symmetric NATs. When two symmetric NAT users try to connect to each other, NAT traversal is impossible and no connection can be made. A workaround is to proxy through the connection through a TURN server. The PeerServer cloud service does not provide a TURN server. You'll have to find your own. You can pass a TURN server into the Peer object options. This will allow your PeerJS app to work seamlessly for this situation

ADD REPLY
0
Entering edit mode
ADD REPLY
0
Entering edit mode

This topic is getting more and more interesting:

https://github.com/feross/webtorrent

ADD REPLY
1
Entering edit mode
10.3 years ago
Daniel ★ 4.0k

I think this is a great idea. I do think that some institutes might block torrent data outright though.

Does anyone know the fidelity of torrents? I know that it should be self checking but is that 100% accurate?

ADD COMMENT
0
Entering edit mode

each piece is protected by a cryptographic hash, altering that is practically unfeasible

ADD REPLY
1
Entering edit mode
10.3 years ago
martenson ▴ 380

Seems working, however only 1 of 3 test torrents will download (the largest one).

see: https://www.dropbox.com/s/knozqs9gitmf6ai/Screenshot%202014-07-15%2016.17.25.png

ADD COMMENT
0
Entering edit mode

Strange may be some sort of traffic filtering or DHCP problem yesterday they worked but today the don't - both of those were shared via wireless computers at PSU.

The large file was shared from my home system.

ADD REPLY

Login before adding your answer.

Traffic: 1529 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6