# Is bbcp a dead project? It only appears to segfault



## incorporeal (May 24, 2018)

sysutils/bbcp seems like a very useful tool. It appears to come up quite often when researching sending large ZFS filesystems at maximum speeds. It appears that the software doesn't function on FreeBSD that well. All of the options and examples I try simply lead to a segfault. I have seen posts at recent as 2017 saying that they are using tools like zrep + bbcp regularly. I'm not sure how they are doing that if the software does not work. The last port patch shows as 20 Jul 2016. 

Examples


```
bbcp -P 2 -s 8 /dev/zero localhost:/dev/null
Password for root@zfsvm1.example.com:
Segmentation fault (core dumped)
```


```
bbcp -4 c -N io "zfs send -R zroot/data@send" zfs1:"zfs recv -d storage/receive"
Segmentation fault (core dumped)
```


```
bbcp -P 2 -s 8 /dev/zero zfs1:/dev/null
Segmentation fault (core dumped)
```


----------



## ShelLuser (May 24, 2018)

Never used it myself, but when in doubt you might want to contact the maintainer first, maybe they can shed some more light on this:


```
peter@zefiris:/usr/ports/sysutils/bbcp#make maintainer
portmaster@BSDforge.com
```
For what's it worth I wouldn't want to rely on this myself, I just noticed that it doesn't even provide a manualpage, which is definitely a mandatory requirement for me.

Even so....  How did you install this?  When I try this on my jail it doesn't dump core at all:


```
root@psi:~ # bbcp -P 2 -s 8 /dev/zero localhost:/dev/null
Password for root@psi.intranet.lan:
bbcp: Protocol not supported creating inet socket
bbcp: Protocol not supported creating inet socket
bbcp: bbcp unexpectedly terminated on localhost
```
... even though I'd imagine that the restrictions within the jail would be enough for the program to go haywire. I don't have a quick test setup available right now and I don't trust this program enough to try it on my main host.


----------



## incorporeal (May 24, 2018)

I installed it with pkg install bbcp. There exists a man page here - https://www.slac.stanford.edu/~abh/bbcp/ which is good enough. It is a very simple tool.


----------



## phoenix (May 25, 2018)

There's a PR for this port (entered by me years ago). There's a minimum set of command-line options that must be set in order for it to work. If you don't use the correct options, it segfaults.

The 2012 package still works (at least on my 10.x systems) as well.

Edit: here's the PR:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197035

Avoid -s and -t options. "-4 c" is required.

Don't use IPv6 (disable it via sysctl).


----------



## incorporeal (May 25, 2018)

phoenix said:


> There's a PR for this port (entered by me years ago). There's a minimum set of command-line options that must be set in order for it to work. If you don't use the correct options, it segfaults.
> 
> The 2012 package still works (at least on my 10.x systems) as well.
> 
> ...



I actually found that using mbuffer provides excellent speeds, basically maxing out the line, and is still well supported and maintained. Would there be any reason at all or feature that bbcp has that would make it worth the hassle over mbuffer?

I am using it for ZFS sends by the way.


----------



## phoenix (May 26, 2018)

We started out using the HPN patches to OpenSSH and the NONE cipher for our ZFS sends. That allowed us to max out a gigabit fibre link between data centres. Then it was dropped from the base OpenSSH install and I didn't feel like compiling OpenSSH (had moved on to binary packages and didn't have a build box setup). So went looking for another solution.

bbcp-2012* was the first one I found, and it worked right away. So we migrated our scripts to use it. Still get an encrypted authentication and max out the gigabit link.

bbcp-2015* gave us issues, but we got a working setup eventually and the older version worked without issues while we figured things out.

Haven't really needed to look for anything else. 

Plain OpenSSH on our systems max out around 400-500 Mbps (regardless of cipher) which isn't sufficient. Have heard mbuffer mentioned by various people on the mailing lists, but never looked into it. Can you share an example command-line for it? And can it saturate a gigabit link on a system where plain OpenSSH can't?

Our only requirements, per se, are encrypted logins and gigabit throughout.


----------



## incorporeal (May 31, 2018)

mbuffer maxes out the line for me but it is not encrypted.  Here is an example command for you.

You must install mbuffer on both the sender and receiver first with

`pkg install mbuffer`

On the receiving machine first

`mbuffer -4 -s 128k -m 1G -I 9999 | zfs receive -F -d tank/receiving`

On the sender once mbuffer is listening above

`zfs send -R zroot/dataset@backup | mbuffer -4 -s 128k -m 1G -O ip-address-of-receiver:9999`


----------



## phoenix (Jun 4, 2018)

Hrm, that could be an issue, having to run a command on the receiving side first before sending the stream.  Would make the scripting a lot more complex to make sure the mbuffer is running correctly before sending the stream (and I send each dataset separately).

The nice thing about bbcp is that it's all controlled by the sender.  You don't have to get/keep anything running on the receiver prior to sending the data.

There also doesn't appear to be any authentication happening with the mbuffer variation, which is something that bbcp provides.


----------



## incorporeal (Jun 6, 2018)

phoenix said:


> Hrm, that could be an issue, having to run a command on the receiving side first before sending the stream.  Would make the scripting a lot more complex to make sure the mbuffer is running correctly before sending the stream (and I send each dataset separately).
> 
> The nice thing about bbcp is that it's all controlled by the sender.  You don't have to get/keep anything running on the receiver prior to sending the data.
> 
> There also doesn't appear to be any authentication happening with the mbuffer variation, which is something that bbcp provides.




What's the easiest way to build the version of bbcp that you use and that works? I'm also wondering how to build this version of bbcp in Poudriere.


----------



## phoenix (Jun 7, 2018)

incorporeal said:


> What's the easiest way to build the version of bbcp that you use and that works? I'm also wondering how to build this version of bbcp in Poudriere.



`pkg install bbcp`


----------



## incorporeal (Jun 8, 2018)

phoenix said:


> `pkg install bbcp`



How would you go about installing bbcp-2012*?


----------



## phoenix (Jun 8, 2018)

incorporeal said:


> How would you go about installing bbcp-2012*?



By searching through the pkg.freebsd.org website to find a repo that includes it, downloading the file from there, and installing if from the command-line via `pkg install /path/to/bbcp-20120520.txz` and making sure that you have the misc/compat10x port installed if using FreeBSD 11.0+.

Note:  I have *never* compiled it from ports; it's always been installed from binary pkg.  I stopped using the ports tree directly years ago.


----------

