# SMB speed issues and how to solve them?



## throb (May 21, 2012)

Hey all,

I have put together a home media server with 16 drives, and using zfs as my choice of file systems. I built a test system previously and it was working well but the new system is not as trouble free. I have been copying to it from a Win7 box and I get 20-30MB/sec from that. Using that same Win7 box I get 100MB/sec to a Thecus NAS box which has FEWER drives in it.

For giggles I took some spare hardware and built an ubuntu box specifically so I could run iperf on it.

*F*rom the ubuntu (as a client) to the freebsd FreeBSD (as a server) I get this:

```
Client connecting to 192.168.3.31, TCP port 5001
TCP window size: 0.02 MByte (default)
------------------------------------------------------------
[  3] local 192.168.3.21 port 46696 connected with 192.168.3.31 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1094 MBytes   109 MBytes/sec
```
So it appears that's pretty damned solid.

However, even copying data from that box to the FreeBSD system is slow.

I ran a *gstat* to see what the system was doing during a copy. I did an rsync between the FreeBSD box and the Thecus NAS.  The drives would sit at 80%+ busy for a bit and then go to zero for several seconds.  Of course this meant that the files (approximately 5000 8MB files) stop copying as well.  It resumes easily enough but there is clearly some kind of bottleneck.  I have a feeling like there is something in the Samba config that's causing trouble.  I am doing all the testing via SMB because that's how I am going to connect the windows box to it.

Any ideas and help are very much appreciated.

Rob


----------



## SirDice (May 21, 2012)

Rebuild samba and turn on WITH_AIO_SUPPORT. That should improve things.


----------



## throb (May 21, 2012)

SirDice said:
			
		

> Rebuild samba and turn on WITH_AIO_SUPPORT. That should improve things.



Thanks for this.  I added 
	
	



```
## add AIO support
aio_load="YES"
```
 to my /boot/loader.conf.

*W*here do I need to set the setting to turn WITH_AIO_SUPPORT to on?

Sorry for the hand holding needed here.  I will learn however 

Rob


----------



## throb (May 21, 2012)

Sort of answering my question:

```
cd /usr/ports/net/samba36
make install clean
```
*C*ompiling now...


----------



## SirDice (May 21, 2012)

throb said:
			
		

> where do I need to set the setting to turn WITH_AIO_SUPPORT to on?




```
cd /usr/ports/net/samba36
make config
# Turn on AIO_SUPPORT
make install clean
```

Unfortunately this option can only be turned on using the port.


----------



## throb (May 21, 2012)

I had to do a *make deinstall* followed by a *make reinstall* then I did the *make config* and *make install clean*, rebooting now.

*T*hanks SirDice!


----------



## throb (May 21, 2012)

Boom! Why is this not enabled by default? What a huge difference it makes!

Many thanks SirDice


----------



## jnbek (May 21, 2012)

if you have portupgrade installed, you can [CMD=]portinstall -C samba36[/CMD] to get the same effect.


----------



## SirDice (May 21, 2012)

Yes, it makes a huge difference :e

I think the reason it's not on by default is because it fails to build on FreeBSD 7.4 (which is still a supported version).


----------



## throb (May 21, 2012)

*T*hanks jnbek for that.  *W*ill know for next time.  I am building another box soon so this comes in handy.

It's a shame that the FreeBSD > 7.4 distributions don't come with the configuration set to turn it on. Seems logical to me but hey, I don't build distributions


----------



## SirDice (May 21, 2012)

There is no 'dist'. All versions of FreeBSD use the exact same ports tree.


----------



## throb (May 21, 2012)

Ah ok.

Oh, I have been watching my rsync and defin*i*tely there is a speed increase.  I of course also have a gstat running and utilization is better there too.  I still get a pause after ~2 seconds of writing to the pool (approximately a 1 second pause) and then writes begin again.

AIO was a good thing for sure.  Where else can I look to try to ferret this out? It's frustrating me to not know how to do this 

Once again, thanks so much for your help!

Rob


----------



## SirDice (May 21, 2012)

You can try setting aio read size and aio write size in smb.conf.

I'm not sure if these are good for you but I have these in smb.conf:

```
socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=65536 SO_SNDBUF=65536
   aio read size = 1
   aio write size = 1
```


----------



## throb (May 21, 2012)

Ok, set that.  I will keep an eye on this to see how this affects things.
It's feeling better so hopefully this sorts it.

thanks!
Rob


----------



## phoenix (May 22, 2012)

You don't mention your pool layout, which can have a huge impact on the theoretical max throughput.


----------



## throb (May 22, 2012)

Here is my pool setup :



```
NAME             STATE     READ WRITE CKSUM
        storage          ONLINE       0     0     0
          raidz2-0       ONLINE       0     0     0
            gpt/pool_a1  ONLINE       0     0     0
            gpt/pool_a2  ONLINE       0     0     0
            gpt/pool_a3  ONLINE       0     0     0
            gpt/pool_a4  ONLINE       0     0     0
            gpt/pool_a5  ONLINE       0     0     0
            gpt/pool_a6  ONLINE       0     0     0
            gpt/pool_a7  ONLINE       0     0     0
            gpt/pool_a8  ONLINE       0     0     0
          raidz2-1       ONLINE       0     0     0
            ada8         ONLINE       0     0     0
            ada9         ONLINE       0     0     0
            ada10        ONLINE       0     0     0
            ada11        ONLINE       0     0     0
            ada12        ONLINE       0     0     0
            ada13        ONLINE       0     0     0
            ada14        ONLINE       0     0     0
            ada15        ONLINE       0     0     0
```

2 vdevs (raidz2) with 8 drives in each.

Rob


----------

