# FreeBSD as NFS server(large scale)



## nizar.abed (Jun 20, 2022)

Hello everyone,

New to FreeBSD...
I'm considering FreeBSD as a NFS server with ZFS.
While I noticed the great features of the latest FreeBSD, including ZFS latest and, I'm trying to setup NFS server for many clients, and while tuning NFS server on FreeBSD(13.1), I couldn't change NFS maxthreads more than 256.
Setting nfs_server_flags = "-u -t -n 4000" in /etc/rc.conf
then 
/etc/rc.d/nfsd restart
I get
nfsd: nfsd count too high 4000; truncated to 256

Trying to adjust under /etc/sysctl.conf
vfs.nfsd.maxthreads=4000
Also doesn't work

. is there anyway to use FreeBSD as NFS server with high number of nfs threads?
. Is FreeBSD the right choice for large scale robust NFS server?

The testing I'm doing are on pretty powerful server:
sysctl hw.model hw.machine hw.ncpu:

hw.model: AMD EPYC 7402 24-Core Processor                
hw.machine: amd64
hw.ncpu: 96

768GB RAM.

Thanks in advance.


----------



## peter-h (Jun 30, 2022)

Why not try it "as is " and slowly increasing maxthreads if performance suffer ?


----------



## Phishfry (Jun 30, 2022)

What is the NFS max thread limit
		

Have you tried modifying source and test?
nfsd.c contains the variable.

Look at this posts PS  too:


> The nfsd.c in head/current/10.0 also has a "--maxthreads" option
> that I don't think is limited to MAXNFSDCNT from a glance at the code.


rmacklem@ is active in NFS and I would ask his opinion.


----------



## zirias@ (Jul 1, 2022)

peter-h said:


> Why not try it "as is " and slowly increasing maxthreads if performance suffer ?


Follow-up question: What should "large scale" mean here?

If you'd expect up to 4000 clients and conclude you'd need 4000 threads for that, it would be a flawed assumption.

Requests don't arrive from all the clients simultaneously, so a single thread can serve multiple clients. There's also a limitation how much parallelism would work fine on your actual storage devices.


----------



## Phishfry (Jul 1, 2022)

I think it would be worth exploring pNFS(4) as well. (Parallel NFS Protocol)
It just hit in 12-RELEASE and was upgraded in 13-RELEASE.



			https://people.freebsd.org/~rmacklem/pnfs-planb-setup.txt
		


It does go against your 'one big NFS machine' philosophy. Distributes the load with parallel servers.



			http://www.pnfs.com/


----------



## Sergei_Shablovsky (Jul 1, 2022)

nizar.abed said:


> Hello everyone,
> 
> New to FreeBSD...
> I'm considering FreeBSD as a NFS server with ZFS.





nizar.abed said:


> . is there anyway to use FreeBSD as NFS server with high number of nfs threads?
> . Is FreeBSD the right choice for large scale robust NFS server?


Why not to use well-reputable FreeNAS ? Also based on FreeBSD...



nizar.abed said:


> The testing I'm doing are on pretty powerful server:
> sysctl hw.model hw.machine hw.ncpu:
> 
> hw.model: AMD EPYC 7402 24-Core Processor
> ...


Really powerful machine.

But for NAS You need care more on:
- best possible RAID/hub card;
- 2 PSU in your server case;
- great online interactive UPS (like Liebert or Emerson made, rack(!) version);
- a lot slots of 3,5” SATA drives;
- a lot of ECC RAM (ChipKill is also great choice);
- better to have active backplane enhancer;


----------



## Phishfry (Jul 1, 2022)

Sergei_Shablovsky said:


> Why not to use well-reputable FreeNAS ? Also based on FreeBSD...


If this guy is managing 4K NFS seats I doubt he needs a web interface.


----------

