# Kerberized NFSv4 -> NFS over TLS on 13.0



## cmoerz (Dec 26, 2021)

Has anyone got a good reference for how to set up krb5p security? I've got NFSv4 running fine without security, but that kerberos setup has been a major pain in the neck. There doesn't seem to be much logging going on, no matter how much -d or -h's I use. It appears, Wireshark is my only utility to figure this out, it feels.

What I've got so far: kdc is running, I can `kinit -k nfs/host` on the client, but I only get "Permission denied" messages on a mount.


```
root@t1:/home/user # mount_nfs -o nfsv4,gssname=nfs@t1.domain,principal=nfs@t1.domain,sec=krb5p svc1.domain:/ /system/nfs/svc1/
mount_nfs: nmount: /system/nfs/svc1: Permission denied
```

This is what `gssd` gives me when I run it in debug mode:

```
gssd_acquire_cred: desired name for host based initiator cred major=0x0 minor=0
gssd_acquire_cred: using keytab entry for nfs/t1.domain, kerberos ret=0
gssd_acquire_cred: done major=0x0 minor=0
gssd_release_name: done major=0x0 minor=0
gssd_import_name: done major=0x0 minor=0
gssd_init_sec_context: done major=0x1 minor=0 uid=0
gssd_release_name: done major=0x0 minor=0
gssd_delete_sec_context: done major=0x0 minor=0
```

Haven't found what "major=0x1" means.

Anyone got any suggestions? I'd appreciate any pointers, even a book would be fine (didn't find anything that seems to cover kerberized NFSv4 on BSD in depth).

Again, `kinit` for a user works fine, the host key works fine with the "-k" flag on the command line. I've also fixed my /etc/hosts file to only include one definitive hostname entry. `hostname -f` successfully returns a FQDN on server and client.

My /etc/krb5.conf (altered to a generic domain name) on the client:

```
[libdefaults]
    default_realm = DOMAIN
[realms]
    DOMAIN = {
        kdc = kdc.domain
        admin_server = kdc.domain
    }
[domain_realm]
    .domain = DOMAIN
```

/etc/rc.conf on client:

```
nfsuserd_enable="YES"
nfscbd_enable="YES"
nfscbd_flags="-P nfs/t1.domain@DOMAIN"
gssd_flags="-h -v -v -v -v"
gssd_enable="YES"
nfsuserd_flags="-domain t1.domain -verbose"
```

/etc/exports on server:

```
V4: /system/nfs -sec=krb5p -network=10.10.0.0/24
```


----------



## T-Daemon (Dec 26, 2021)

cmoerz said:


> /etc/exports on server:
> 
> ```
> V4: /system/nfs -sec=krb5p -network=10.10.0.0/24
> ```


In this form no file system is exported.

exports(5) says:

```
The third form has the string    ``V4:''    followed by a
     single absolute path name,    to specify the NFSv4 tree root.     This line
     does not export any file system, but simply marks where the root of the
     server's directory    tree is    for NFSv4 clients.  The    exported file systems
     for NFSv4 are specified via the other lines in the    exports    file in    the
     same way as for NFSv2 and NFSv3.
```

For application see EXAMPLES section at the beginning and the end.


----------



## cmoerz (Dec 26, 2021)

Good point. Somehow, I missed that in the man page.

I've now altered my /etc/exports file to

```
V4: /system/nfs -sec=krb5p -network=10.10.0.0/24
/ -network=10.10.0.0/24 -sec=krb5p
/system/nfs -network=10.10.0.0/24 -sec=krb5p
```

Still getting

```
mount_nfs: nmount: /system/nfs/svc1: Permission denied
```
unfortunately. I suppose I'll be giving up on kerberos and will stick to unencrypted sec security.

I'm beginning to wonder whether this is such a special thing that simply nobody is using anyways or if I'm just too stupid to figure this out.
Is it just me or is this whole NFS thing seriously lacking in the logging department?

Running `tcpdump` and reading the dump in wireshark shows, that I'm getting a NFS V4 NULL reply that says

```
Reject State: AUTH_ERROR (1)
Auth State: client must begin new session (2)
```

Not sure, how to fix this. Not even the slightest idea where to start with this...


----------



## BjarneB (Dec 27, 2021)

I spend a lot of time on nfsv4/kerberos a couple of years ago setting up linux clients against a hitachi hnas storage and my advice would be,
unless you really really really really need end-to-end encryption on the transport level, forget it.
It is very complex and shaky to setup and it is a nightmare to maintain and keep running.
keytabs that expire, dependency of kerberos servers, very strict access mapping, no root mapping.
Also a lot of the involved software which was supposed to run on the nfs shares, where not build for that kind of strictness.
After a period of testing we abandoned kerberos for NFS.

One issue we spend a lot of time on  was id mapping. Unless that is correct nothing will work
I just checked my notes, but I am afraid I don't have much, but a couple of links that might help:






						RHEL7: Use Kerberos to control access to NFS network shares. - CertDepot
					

Here you will find RHEL 7 instructions to control access to NFS network shares with Kerberos.



					www.certdepot.net
				








						Explain like I’m 5: Kerberos
					

Explain like I’m 5 years old: Kerberos – what is Kerberos, and why should I care? While this topic probably can not be explained to a 5 year-old and be understood, this is my attempt at defragmenting documentation with some visual aids and digestible language. In a nutshell Basically, Kerberos...




					www.roguelynn.com
				





			KerberizedNFS - FreeBSD Wiki
		


One of them is linux focused, but I don't see any major  difference to FreeBSD, the concept is the same.


----------



## T-Daemon (Dec 27, 2021)

If KerberizedNFS is unreliable, then give NFS over TLS a shot. It was introduced in 2020-08-22 to 13.0-CURRENT then.

At the present 13.0-STABLE and 14.0-CURRENT have it in base.

13.0-RELEASE has the kernel support ( `options   KERN_TLS` in GENERIC), but no server and client daemons. Those are provided by sysutils/nfs-over-tls.

The port needs to be build with security/openssl-devel installed and  `DEFAULT_VERSIONS+= ssl=openssl-devel` set in /etc/make.conf.

Enable kernel TLS: `sysctl kern.ipc.tls.enable=1`

See mount_nfs(8) and exports(5) for tls option.

Start the daemons with rc command scripts tlsclntd and tlsservd from  /etc/rc.conf.

Before starting tlsservd, some pem certificates need to be created (see nfs-over-tls-setup.txt).

Tested it myself. So far the rpc.tlsclntd(8) daemon is running on 13.0-RELEASE, rpc.tlsservd(8) complains about missing pem certificates. I hadn't time to create them, not sure when I can find the time to test further, but wanted to share what I have so far.


----------



## cmoerz (Dec 27, 2021)

Thanks, T-Daemon. I've started looking at sysutils/nfs-over-tls; it seems to already work with regular security/openssl with the make.conf options you mentioned.
I'm in the middle of trying things out, but it certainly looks a lot less painful and way more promising than kerberos.


----------



## cmoerz (Dec 27, 2021)

Gaah, I jinxed it.

So I have an nfs4 share working with sec=sys. I've got rpc.tlsservd running on the server, rpc.tlsclntd on the client. I attempt a mount with -tls and - again: Permission denied.

However, on the server, I just found some strange messages

```
Dec 27 18:51:03 svc1 rpc.tlsservd[45440]: ktls not working
Dec 27 18:51:03 svc1 syslogd: last message repeated 2 times
Dec 27 18:53:18 svc1 syslogd: last message repeated 3 times
```
Same on the client, it turns out:

```
Dec 27 18:51:03 svc2 rpc.tlsclntd[8011]: ktls not working
Dec 27 18:51:03 svc2 syslogd: last message repeated 2 times
Dec 27 18:53:17 svc2 syslogd: last message repeated 3 times
```

I suppose, I will fall back onto security/openssl-devel now.


----------



## Jose (Dec 27, 2021)

Awright, I have not actually tried this, and am going from reading the man page. Try

```
V4: /system -sec=krb5p -network=10.10.0.0/24
/system/nfs -sec=krb5p
```
And mount it like this

```
mount server:/nfs /mnt
```



T-Daemon said:


> If KerberizedNFS is unreliable, then give NFS over TLS a shot. It was introduced in 2020-08-22 to 13.0-CURRENT then.


Kerberos provides secure authentication, TLS encrypts the connection with the server. TLS will use the old insecure UID-based authentication scheme (I think they call it "sys" auth now) even though the message stream will be encrypted. It's not hard to guess a UID since they're small natural numbers, and they are usually within a standard range. Also, UIDs in Freebsd appear to be 32-bit signed integers. A brute force attack on this authentication method is trivial.



T-Daemon said:


> Tested it myself. So far the rpc.tlsclntd(8) daemon is running on 13.0-RELEASE, rpc.tlsservd(8) complains about missing pem certificates. I hadn't time to create them, not sure when I can find the time to test further, but wanted to share what I have so far.


You can do client auth with TLS, but then the client will have to present a certificate (see the `tlscert` and `tlscertuser` documentation in the man page). This is probably attractive for small setups, but you would have to give every client a different certificate, and would have to rotate those certificates periodically in order to be truly secure. This scheme is usually called mTLS, for "mutual TLS". I don't know why they don't refer to it that way in the man page.


----------



## cmoerz (Dec 27, 2021)

Well, from the looks of it, TLS should be easier to set up for a cross-server set up like in my case. Unfortunately, I'm stuck in another dead end, it seems.

I've used security/easy-rsa for setting up certificates, got the rpc.tlsservd and rpc.tlsclntd daemons running, but I keep getting that

```
rpc.tlsservd[45440]: ktls not working
```

Interestingly, the daemon.log gives me further details from the server daemon, that appear to be fine:

```
Dec 27 20:35:41 svc1 rpc.tlsservd[39645]: rpctlsd_connect_svc: started
Dec 27 20:35:41 svc1 rpc.tlsservd[39645]: rpctls_server: BIO_get_ktls_send=0
Dec 27 20:35:41 svc1 rpc.tlsservd[39645]: rpctlssd_connect_svc: succeeded flags=0x11
Dec 27 20:35:41 svc1 rpc.tlsservd[39645]: rpctlssd_disconnect fd=8 closed
```

I've got /boot/loader.conf appended with

```
ktls_ocf_load="YES"
aesni_load="YES"
```
and /etc/sysctl.conf with

```
security.bsd.see_other_uids=0
security.bsd.see_other_gids=0
security.bsd.see_jail_proc=0
security.bsd.unprivileged_read_msgbuf=0
security.bsd.unprivileged_proc_debug=0
kern.randompid=1
vfs.zfs.min_auto_ashift=12

hw.mds_disable=1
hw.spec_store_bypass_disable=1
hw.ibrs_disable=1
machdep.nmi_flush_l1d_sw=1
vm.pmap.allow_2m_x_ept=1
machdep.mitigations.rngds.enable=1
kern.elf32.aslr.enable=1
kern.elf64.aslr.enable=1
kern.elf32.nxstack=1
kern.elf32.allow_wx=0
kern.elf64.allow_wx=0
kern.ipc.tls.enable=1
kern.ipc.mb_use_ext_pgs=1
```

My /etc/rc.conf:

```
clear_tmp_enable="YES"
syslogd_flags="-ss"
sendmail_enable="NONE"
hostname="svc1.domain"
ifconfig_vtnet0="inet 10.10.0.66 netmask 255.255.255.0"
ifconfig_vtnet0_alias0="inet 10.10.0.89 netmask 255.255.255.255"
ifconfig_vtnet1="inet 10.10.1.2 netmask 255.255.255.0"
ifconfig_vtnet2="inet 10.10.2.1 netmask 255.255.255.224"
defaultrouter="10.10.0.1"
sshd_enable="YES"
ntpdate_enable="YES"
powerd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"
zfs_enable="YES"
auditd_enable="YES"
jail_enable="YES"
kdc_enable="YES"
kdc_flags="--addresses=10.10.0.89"
pf_enable="YES"
nfsuserd_enable="YES"
nfs_server_enable="YES"
nfs_server_flags="-t -u -h 10.10.2.1 -V svc1.nfs.domain"
nfsv4_server_only="YES"
rpcbind_flags="-h 10.10.2.1"
mountd_flags="-r -S -R"
nfsuserd_flags="-domain domain -verbose"
gssd_enable="YES"
gssd_flags="-h -v"
rpcbind_enable="YES"
tlsservd_enable="YES"
#tlsservd_flags="-l /system/ca/pki/ca.crt -n nfs.domain-r /system/ca/pki/crl.pem -v"
tlsservd_flags="-v"
```

The zfs share is enabled as well. I've made sure to have DNS and reverse DNS working properly for the server and the client.

I'm stumped. Anybody got any suggestions? Except for the certificate creation, I've followed this:



			https://people.freebsd.org/~rmacklem/nfs-over-tls-setup.txt
		


Now I'm confused why it's not working - even when I'm leaving out the certificate arguments.

After some additional sleuthing, I've come across this code section:

```
#ifdef BIO_get_ktls_send
        /* Check to see that ktls is working for the connection. */
        ret = BIO_get_ktls_send(SSL_get_wbio(ssl));
        rpctls_verbose_out("rpctls_server: BIO_get_ktls_send=%d\n", ret);
        if (ret != 0) {
                ret = BIO_get_ktls_recv(SSL_get_rbio(ssl));
                rpctls_verbose_out("rpctls_server: BIO_get_ktls_recv=%d\n",
                    ret);
        }
#else
        ret = 0;
#endif
        if (ret == 0) {
                if (rpctls_debug_level == 0)
                        syslog(LOG_ERR, "ktls not working");
                else
                        fprintf(stderr, "ktls not working\n");
                /*
                 * The handshake has completed, so all that can be
                 * done is disable the connection.
                 */
                *flags |= RPCTLS_FLAGS_DISABLED;
        }
```

That's where things break. So, now I'll need to figure out, what BIO_get_ktls_send is doing and why it fails. Or maybe SSL_get_wbio is the culprit...

Further update after some digging: That method checks whether it's the kernel doing the TLS and it fails, because it's not. I unfortunately don't know why. Compiling base with

```
WITH_OPENSSL_KTLS=1
```
makes it work on the server side now. Client still broken though, with a similar output. However, on the client side I'd rather stick with openssl from ports because the documentation says changing base openssl may break ports like apache (and I assume also other web servers).


----------



## T-Daemon (Dec 29, 2021)

Jose said:


> V4: /system -sec=krb5p -network=10.10.0.0/24
> /system/nfs -sec=krb5p


The `-network` option in the `V4:` line has no effect:
exports(5)

```
... The third form has the
     string ``V4:'' followed by a single absolute path name, to specify the
     NFSv4 tree root
     ...
                                         ... .For the NFSv4 tree root, the
     only options that can be specified in this section are ones related to
     security: -sec, -tls, -tlscert and -tlscertuser.
```



Jose said:


> Kerberos provides secure authentication, TLS encrypts the connection with the server. TLS will use the old insecure UID-based authentication scheme (I think they call it "sys" auth now) even though the message stream will be encrypted. It's not hard to guess a UID since they're small natural numbers, and they are usually within a standard range. Also, UIDs in Freebsd appear to be 32-bit signed integers. A brute force attack on this authentication method is trivial.


Thanks for the provided explanation. I don't have much knowledge about networks and their possible security mechanisms  and can't comment the technical part, but as a layman I ask myself, if a brute force attack on a authentication method you described is trivial then why implement it at all.

Tracking the commit logs in the FreeBSD GIT repository, there is significant effort invested in implementing NFS over TLS on FreeBSD. Why would they invest that much effort in a weak security implementation?

There is article Using TLS to Improve NFS Security by Rick Mecklem (PDF) from the May/June 2021 FreeBSD Journal, describing the implementation technically.



Jose said:


> You can do client auth with TLS, but then the client will have to present a certificate (see the `tlscert` and `tlscertuser` documentation in the man page).


Not necessarily. From exports(5):

```
The -tls, -tlscert and -tlscertuser export options are used to require
     the client to use TLS for the mount(s) per RFC NNNN.  For NFS mounts
     using TLS to work, rpc.tlsservd(8) must be running on the server.

           -tls requires that the client use TLS.
```

and linked article:

```
tls - Indicates that the client must use NFS over TLS, but is not required
     to present any X.509 certificate to the server during TLS handshake.
```


----------



## T-Daemon (Dec 29, 2021)

Update on NFS over TLS.

I couldn't make it work on 13.0-RELEASE. I don't know what's missing in -RELEASE but it works on -STABLE. I've created two VM's (13.0-STABLE-20211223) with following configuration:

Server:

/boot/loader.conf

```
...
ktls_ocf_load="YES"
```

/etc/rc.conf

```
...
tlsservd_enable="YES"
```

/etc/sysctl.conf

```
...
kern.ipc.tls.enable=1
kern.ipc.mb_use_ext_pgs=1
```

/etc/rpc.tlsservd/{cert.pem, certkey.pem}

/etc/exports (no ZFS sharenfs setting):

```
V4: /usr/storage -tls

/usr/storage/data -mapall=root  -network x.x.x.x/24
```


Client:

/boot/loader.conf

```
...
ktls_ocf_load="YES"
```

/etc/rc.conf

```
...
tlsclntd_enable="YES"
```

/etc/sysctl.conf

```
...
kern.ipc.tls.enable=1
kern.ipc.mb_use_ext_pgs=1
```

No certificates on client.

Mount without tls:

```
# mount -o nfsv4 x.x.x.x:/data /mnt
mount_nfs: nmount: /mnt: Permission denied
```

Mount with tls is successful, directory accessible for operations:

```
# mount -o nfsv4,tls x.x.x.x:/data /mnt
```

I haven't tested `-tlscert` `-tlscertuser` with X.509 certificates yet.

Can somebody point to a utility to verify NFS is over TLS?


----------



## covacat (Dec 29, 2021)

cat some text files and tcpdump


----------



## T-Daemon (Dec 29, 2021)

I need a little help here.

 Client `cat /mnt/<file>`, `tcpdump` on second console.

192.168.42.42 is the client, 192.168.42.206 the server, 192.168.42.129 the router.


```
18:44:00.121268 IP client.ftps-data > nfstlsserv.nfsd: Flags [P.], seq 3071514930:3071515167, ack 2818151329, win 4352, options [nop,nop,TS val 4210280629 ecr 621431506], length 237
18:44:00.121634 IP nfstlsserv.nfsd > client.ftps-data: Flags [P.], seq 1:326, ack 237, win 29128, options [nop,nop,TS val 621440920 ecr 4210280629], length 325
18:44:00.121722 IP client.ftps-data > nfstlsserv.nfsd: Flags [P.], seq 237:502, ack 326, win 4352, options [nop,nop,TS val 4210280629 ecr 621440920], length 265
18:44:00.121915 IP nfstlsserv.nfsd > client.ftps-data: Flags [P.], seq 326:863, ack 502, win 29128, options [nop,nop,TS val 621440920 ecr 4210280629], length 537
18:44:00.121968 IP client.ftps-data > nfstlsserv.nfsd: Flags [P.], seq 502:739, ack 863, win 4352, options [nop,nop,TS val 4210280629 ecr 621440920], length 237
18:44:00.122147 IP nfstlsserv.nfsd > client.ftps-data: Flags [P.], seq 863:1188, ack 739, win 29128, options [nop,nop,TS val 621440920 ecr 4210280629], length 325
18:44:00.122195 IP client.ftps-data > nfstlsserv.nfsd: Flags [P.], seq 739:1024, ack 1188, win 4352, options [nop,nop,TS val 4210280629 ecr 621440920], length 285
18:44:00.122376 IP nfstlsserv.nfsd > client.ftps-data: Flags [P.], seq 1188:1413, ack 1024, win 29128, options [nop,nop,TS val 621440920 ecr 4210280629], length 225
18:44:00.122516 IP client.ftps-data > nfstlsserv.nfsd: Flags [P.], seq 1024:1253, ack 1413, win 4352, options [nop,nop,TS val 4210280629 ecr 621440920], length 229
18:44:00.122686 IP nfstlsserv.nfsd > client.ftps-data: Flags [P.], seq 1413:1722, ack 1253, win 29128, options [nop,nop,TS val 621440920 ecr 4210280629], length 309
18:44:00.122745 IP client.ftps-data > nfstlsserv.nfsd: Flags [P.], seq 1253:1490, ack 1722, win 4352, options [nop,nop,TS val 4210280629 ecr 621440920], length 237
18:44:00.122904 IP nfstlsserv.nfsd > client.ftps-data: Flags [P.], seq 1722:1875, ack 1490, win 29128, options [nop,nop,TS val 621440920 ecr 4210280629], length 153
18:44:00.163203 IP client.ftps-data > nfstlsserv.nfsd: Flags [.], ack 1875, win 4352, options [nop,nop,TS val 4210280672 ecr 621440920], length 0
18:44:00.773687 IP client.50666 > 192.168.42.129.domain: 24737+ PTR? 42.42.168.192.in-addr.arpa. (44)
18:44:00.780253 IP 192.168.42.129.domain > client.50666: 24737* 1/0/0 PTR client. (64)
18:44:00.780457 IP client.23831 > 192.168.42.129.domain: 13213+ PTR? 206.42.168.192.in-addr.arpa. (45)
18:44:00.786238 IP 192.168.42.129.domain > client.23831: 13213* 1/0/0 PTR nfstlsserv. (69)
18:44:01.792699 IP client.21320 > 192.168.42.129.domain: 5862+ PTR? 129.42.168.192.in-addr.arpa. (45)
18:44:01.799090 IP 192.168.42.129.domain > client.21320: 5862 NXDomain 0/0/0 (45)
```


----------



## covacat (Dec 29, 2021)

use tcpdump  -X and see if you can read it
or -w file.out  and then strings the file


----------



## T-Daemon (Dec 29, 2021)

covacat said:


> use tcpdump -X and see if you can read it


I can read it, but understand not so much. nfstlsserv is the hostname of the server.



			https://termbin.com/5ry9


----------



## covacat (Dec 29, 2021)

T-Daemon said:


> I can read it, but understand not so much. nfstlsserv is the hostname of the server.
> 
> 
> 
> https://termbin.com/5ry9


yeah, does not look like clear text  so it passes the tls-eyetest-tm


----------



## cmoerz (Dec 29, 2021)

Additional tidbit: I got in touch with Rick Macklem. He let me know that sysutils/nfs-over-tls only works on 13 RELEASE, not on STABLE.
Addendum after reading T-Daemon's post, he might have mixed the both up.

I've tried it on release and ran a `tcpdump` to check the result in `wireshark`. It certainly looked like some kind of encryption was going on, but the client simply got permission errors and failed to produce a working result nonetheless.

If you run `rpc.tlsservd` or `rpc.tlsclntd` with the -v parameter, you'll very likely get the same output as me in /var/log/daemon.log.


----------



## Jose (Dec 29, 2021)

T-Daemon said:


> The `-network` option in the `V4:` line has no effect:
> exports(5)
> 
> ```
> ...


Yeah, I saw that. However later in the man page we find

```
/private -sec=krb5i
       /secret -sec=krb5p
       V4: /   -sec=krb5:krb5i:krb5p -network 131.104.48 -mask 255.255.255.0
       V4: /   -sec=sys:krb5:krb5i:krb5p grumpy.cis.uoguelph.ca
```
Those lines are explained like this somewhat later down the man page


> For the experimental server, the NFSv4 tree is rooted at ``/'', and any
> client within the 131.104.48 subnet is permitted to perform NFSv4 state
> operations    on the server, so long as valid    Kerberos credentials are pro-
> vided.  The machine grumpy.cis.uoguelph.ca    is permitted to    perform    NFSv4
> ...


I figure at worst it'll do no damage. That man page could use some love. Maybe we can get Grahamperrin or Erichans interested in cleaning it up.


T-Daemon said:


> Tracking the commit logs in the FreeBSD GIT repository, there is significant effort invested in implementing NFS over TLS on FreeBSD. Why would they invest that much effort in a weak security implementation?


I didn't say it was weak. I said encryption and authentication are different things. Both should be strong. NFS over TLS provides strong encryption, but is silent about authentication.


T-Daemon said:


> Not necessarily. From exports(5):
> 
> ```
> The -tls, -tlscert and -tlscertuser export options are used to require
> ...


If the client does not present a certificate, it is not authenticated in any way. Any client, including an attacker can connect to your NFS server. It will be difficult to analyze the attack since it will be encrypted securely.

EDIT: The `tls` option makes it so the server presents a certificate. Therefore clients can verify the server they're connecting to, and terminate the connection if the certificate the server presents is not valid. However, the server accepts connections from all clients.

Authentication is what you  get from a client certificate. If properly configured, the client will have the private key that allows it to create signatures that can be verified with its certificate. If all clients use the same certificate, it is only as secure as the least secure client.

Certificates are unnecessarily complicated and confusing. This is the best explanation of how they work I've ever seen:








						Everything you should know about certificates and PKI but are too afraid to ask
					

Everything you should know about certificates and public key infrastructure (PKI) but are too afraid to ask.



					smallstep.com


----------



## T-Daemon (Dec 31, 2021)

cmoerz said:


> I've tried it on release and ran a `tcpdump` to check the result in `wireshark`. It certainly looked like some kind of encryption was going on, but the client simply got permission errors and failed to produce a working result nonetheless.
> 
> If you run `rpc.tlsservd` or `rpc.tlsclntd` with the -v parameter, you'll very likely get the same output as me in /var/log/daemon.log.


.
Exactly what I observed. On 13.0-RELEASE with port sysutils/nfs-over-tls installed, client gets "permission denied" when attempting to mount the NFS share over TLS.

On 13.0-STABLE all works well, with daemons rpc.tlsservd(8) on server and rpc.tlscnltd(8) on client from base (not from port). No output in /var/log/daemon.log with the `-v` flag set.


----------



## T-Daemon (Dec 31, 2021)

Jose said:


> That man page could use some love.


It sure does. 



Jose said:


> Maybe we can get _*[FONT=monospace]Grahamperrin[/FONT]*_ or _*[FONT=monospace]Erichans[/FONT]*_ interested in cleaning it up.


After new year, when time permits, I will set up a kerberos server to check out which statement is correct. The one in the DESCRIPTION regarding the NFSv4 tree roots only options being security related or the one in the EXAMPLES section, permitting V4: to set other options as well.

What the outcome will be, I will propose a patch in a PR.



Jose said:


> I didn't say it was weak. I said encryption and authentication are different things. Both should be strong. NFS over TLS provides strong encryption, but is silent about authentication.


A missunderstanding on my part. When you said


Jose said:


> .. TLS encrypts the connection with the server. TLS will use the old insecure UID-based authentication scheme (I think they call it "sys" auth now) even though the message stream will be encrypted. It's not hard to guess a UID since they're small natural numbers, and they are usually within a standard range. Also, UIDs in Freebsd appear to be 32-bit signed integers. A brute force attack on this authentication method is trivial.


it sounded to me TLS can be compromised with little effort. 



Jose said:


> If the client does not present a certificate, it is not authenticated in any way. Any client, including an attacker can connect to your NFS server. It will be difficult to analyze the attack since it will be encrypted securely.
> 
> EDIT: The `tls` option makes it so the server presents a certificate. Therefore clients can verify the server they're connecting to, and terminate the connection if the certificate the server presents is not valid. However, the server accepts connections from all clients.
> 
> ...



Thank you again for the explanation and the link.


----------



## covacat (Dec 31, 2021)

from what i understood from exports(5) krb5p will provide packet encryption and krb5i packet signing so it can replace tls


----------



## Jose (Dec 31, 2021)

T-Daemon said:


> After new year, when time permits, I will set up a kerberos server to check out which statement is correct. The one in the DESCRIPTION regarding the NFSv4 tree roots only options being security related or the one in the EXAMPLES section, permitting V4: to set other options as well.


I suspect they're both correct. If I were to summarize the man page, each line in an export file has three fields:

The literal string "V4:" followed by a path *or* one or more paths. The first form specifies the root of the NFS v4 hierarchy. The second form is used to export filesystems using multiple versions of the NFS protocol, including v4.
Export options. For lines that start with V4 only *-sec*, *-tls*, *-tlscert* and *-tlscertuser *are allowed in this second field. The other options (maproot, etc.) are only allowed in lines that do not start with V4.
The host set to which the line applies. One of the ways to do this is to specify an Internet subnetwork using a network and network mask. This field can be present in any kind of line, but is not required for lines that start with V4.



T-Daemon said:


> A missunderstanding on my part. When you said
> 
> it sounded to me TLS can be compromised with little effort.


Yeah, that was poorly worded. TLS has nothing to do with it. The client says "hey, I'm connecting as user with id 1001", and the server just believes it. This "system" auth works the same way with or without TLS and is horrendously insecure.

You can imagine what happens when the client says "I'm user with id 0". Believe it or not, that was allowed for a long time. Getting rid of this behavior was called "squashing root". It looks like we're using "maproot" now.



T-Daemon said:


> Thank you again for the explanation and the link.


You're welcome! I'm glad you found it useful.


----------



## Andriy (Jan 2, 2022)

Jose said:


> This "system" auth works the same way with or without TLS and is horrendously insecure.


Unless TLS is configured to require trusted certificate(s).


----------



## Jose (Jan 2, 2022)

Andriy said:


> Unless TLS is configured to require trusted certificate(s).


Which is essentially what I said here and here. I would argue that that is not "system" auth, however. The client is authorized by the certificate it presents and not by the UID it claims to have.


----------



## rmacklem (Feb 26, 2022)

I do not normally participate in these forums, so using the
email lists like freebsd-fs@ will normally get my attention.
However, I thought I would post here, with a few possibly
useful bits...

For some time, there has been a problem with the gssd daemon,
where it needs to be restarted after booting, to get it to work.
This simple patch for /etc/rc.d/gssd seems to fix the problem.


			https://people.freebsd.org/~rmacklem/gssd-startup.patch
		


There is also a document that might be useful.


			https://people.freebsd.org/~rmacklem/nfs-krb5-setup.txt
		

It is a work in progress, so feel free to email patches to
rmacklem@uoguelph.ca.

W.r.t. NFS-over-TLS...
- The specific use case of tlscertuser performs all RPCs
  as the user represented by the client's X.509 certificate
  and makes the certificate perform user authentication
  for one user.
The nfs-over-tls in ports (version 1.1) needs this simple
patch to make it work for newer versions of OpenSSL.


			https://people.freebsd.org/~rmacklem/tlsport.patch
		


The code in main, stable/13 (and therefore 13.1) is already
patched and everything should be built in 13.1, so the port
will not be needed.


----------



## mnd999 (May 21, 2022)

I also found this gssd issue the hard way. Is there any reason this can't be merged?


----------



## Jose (May 22, 2022)

mnd999 said:


> I also found this gssd issue the hard way. Is there any reason this can't be merged?


That's a question for rmacklem and he said he prefers to communicate over the freebsd-fs mailing list.


----------

