# NFS cache misses after upgrading to 11.1 from 10.3



## nielsk (Apr 12, 2018)

Recently I upgraded a machine from 10.3-Prerelease to 11.1-release. Since then I see a huge difference in nfs cache misses, especially getattr-misses compared to identical configured machines that are still on 10.3. When I do an nfsstat -m the options are identical.
This leads too less performance and less happy end users. Is there any regression known? Any defaults that changed? Anything else I should know about that could lead to this behavior?

I am grateful for any hint.


----------



## ShelLuser (Apr 12, 2018)

What NFS daemon are you using? The one in the base system or the one provided through the ports collection? In addition to that: if you're using the port, how did you install it? Using the regular build procedure or a build program (Portmaster, Synth, etc.) or binary packages?

Also: how is the FreeBSD version set up, and how did you perform the upgrade? Did you build the base system yourself or did you rely on freebsd-update?


----------



## nielsk (Apr 12, 2018)

There is an NFS daemon in the ports? All systems are NFS clients which are mounting an NFS provided by a linux-server. I use the client provided by the base system (so it is just an fstab-entry which is used for mounting the export)

The old system (10.3-prerelease) is a generic kernel built on the system which has only TCP_FASTOPEN as an additional option added. The upgrade went by building a generic kernel and then upgrade via freebsd-update to 11.1p8-RELEASE. That's how I switched all my systems so far from STABLE to RELEASE. This system was on a PRERELEASE apparently (I inherited those systems).
ports on the old systems are built via portmaster, ports on the new systems are built on a built-host with poudriere.


----------



## ShelLuser (Apr 12, 2018)

To my knowledge you cannot upgrade STABLE (a developer snapshot) using freebsd-update. The only way to upgrade from STABLE is by building the world from source. So if you somehow did do something like that then I can only wonder what might have broken during the process. Not saying that it did, but... the scenario sounds a bit odd to me.

Anyway, when I check the release notes for 10.4 and 11.1 then I do spot some changes within the NFS client. See 8.1 in both release notes. However, I have no idea if that could have influenced things in the way you described here.


----------



## nielsk (Apr 12, 2018)

I did a lot of research before I did it, but you can upgrade stable to release using freebsd-update. It is tricky but possible (you need to build a generic kernel and full world, nothing stripped down, then you can call freebsd-update like "UNAME_r=10.3-RELEASE freebsd-update -r 11.1-RELEASE upgrade" (this works in bash not tcsh). And then it will upgrade. I upgraded before that machine like a dozen servers from 11.0-stable to 11.1-Release. You cannot do this step from 11.0-stable to 11.0-release but need an actual version difference like 11.0 -> 11.1 or 10.3 -> 11.1). But since this was Prerelease, I do not even think that it was stable but as the name suggests "Prerelease" with a custom kernel.

NFS4ERR_BAD_SESSION is only applicable to NFSv4 afaik and we use NFSv3. And the ERESTART/sosend-thingy I don't actually know. There is a mailing list-thread about it and I do not understand it. But my guess is that this shouldn't cause caching-problems.


----------



## robroy (Apr 12, 2018)

nielsk, maybe you don't feel like doing this, yet perhaps reproducing the symptom outside of your current environment could help to narrow things down.

I guess I'd personally make two VMs, and fresh-install 10.3-RELEASE in one, and 11.1-RELEASE in the other, and see if you observe the same NFS server difference between the two.  I "get" that this may be way easier said than done.

And I suppose you've already thought of going to 10.4-RELEASE, in case 11.x really has regressed with how it handles your scenario.


----------



## nielsk (Apr 12, 2018)

Yeah…this is way way easier than done. Because I would need to create a test environment which is more complex than just installing two VMs. I didn't even thought yet of going to 10.4-Release because I cannot upgrade another server without risking that maybe 10.4 shows the same behavior. The test machine where I did the whole upgrade which (also includes an upgrade of PHP to PHP7.1) is a VM  but doesn't use the NFS, showed an increase in performance of 30% in comparison to before the upgrade while this machine which is bare-metal and uses the NFS shows a decrease in performance.


----------

