I've read a few answers concerning the transfer of large data quickly and reliably across the internet. I've not been able to find a tool which combines these features. Does anyone know if a combination of options in the commonly used tool achieves this?
- Multiple TCP streams or UDP for very fast transfer of bulk data
- Similarly sensible re disk writes, threads, poll/select and copying to get stuff onto disk quickly
- Checking of ownership and permissions as well as checksumming at both ends
- Handles multiple small files as efficiently as very large files
- Can run "rsync-style", only transferring diffs when appropriate
- Has good security integrity and authenticity guarantees (secrecy not required)
- good quality linux server and client with robust error detection and reporting.
I've had a look at rsync, fdt, bbcp, unison, aspera, udt/udr, &c, and all seem to offer a subset of these features?
Obviously, through a combination of tools and a load of glue scripts I could achieve this with existing tools, but before going to the effort, if it's just a magic combination of parameters, do let me know!