# Timeouts when sending data to a service inside a jail



## ShelLuser (Oct 6, 2013)

Hi gang,

I'm facing a problem I can't get my fingers behind. I've upgraded my backup server to FreeBSD 9.2 and I'm using this server to keep data "off server" (backups go "off server" by copying them onto the backup server, which is basically a separate VPS with the same hosting provider, and "off site" by copying them from the backup server to a remote location).

In order to enhance my security I've decided that I want to set up a jail to serve as the front-end between my regular servers and the backup server. It's nearly impossible to use NFS from inside a jail but it turns out that services such as FTP and Samba run quite well.

I have a dedicated network device for the jail (vtnet1) and the only IP address which is assigned to it is the IP address of the jail itself, an IPv4 address which sits in the private range and falls within the network I've setup. For the record; obviously it's also being used inside a private network, hence the class C addresses.

I've also set these settings:


```
security.jail.allow_raw_sockets=1
security.jail.sysvipc_allowed=1
```
...in /etc/sysctl.conf and also made sure to set jail_sysvipc_allow inside my /etc/rc.conf file.

All servers can access the jail without problems. However, the moment I'm trying to send a file which is roughly larger than 1kB the whole action stalls. No matter if I'm using FTP, smbfs or even SSH.

If I try the same action on the host itself (using the same network device and IP address) everything works as expected.

I can rule out a firewall issue, I've tried to set / unset forwarding, but I just can't figure out what is going wrong.

The only thing I'm now considering if it might be related to me disabling the bpf device in my custom kernel setup. So I don't have /dev/bpf on my systems. This also disabled the use of tcpdump.

Could that be related?


----------



## fbsd1 (Oct 7, 2013)

Enabling those two nibs is NOT enhancing your security; it's doing the complete opposite. They should never be used in a production environment. Your bpf question is easy to test, just compile a kernel with it enabled.

Do you have a alias for the jails IP address?


----------



## ShelLuser (Oct 8, 2013)

Well, so far I can easily conclude that /dev/bpf has no (noticeable) influence on jail functionality and that there seems to be something odd going on. Possibly related to the known issues with if_vtnet but it's way too soon for me to draw such conclusions.

Either way something weird seems to be going on.

For example, when connecting to an FTP daemon inside the jail and then downloading a file several packets are being detected by tcpdump but for reasons unknown to me right now they're not being acknowledged or detected:


```
17:43:32.658387 IP gadget.dogma.57803 > central.dogma.ftp: Flags [.], ack 763, win 1040, options [nop,nop,TS val 358734602 ecr 1719980930], length 0
17:43:32.855295 IP central.dogma.55146 > gadget.dogma.44004: Flags [.], seq 1:1449, ack 1, win 1040, options [nop,nop,TS val 1755435688 ecr 358734502], length 1448
17:43:33.255304 IP central.dogma.55146 > gadget.dogma.44004: Flags [.], seq 1:1449, ack 1, win 1040, options [nop,nop,TS val 1755436088 ecr 358734502], length 1448
17:43:33.855257 IP central.dogma.55146 > gadget.dogma.44004: Flags [.], seq 1:1449, ack 1, win 1040, options [nop,nop,TS val 1755436688 ecr 358734502], length 1448
```
(where central.dogma is the jail and gadget.dogma a remote server).

Despite the packets the end result is a timeout. It doesn't matter if I initiate the connection from a remote server of from within the jail, nor does it matter if I upload or download files.

Either way, the tcpdump logs give me something new to study so that's where I'll be focussing my attention on for now.


----------



## junovitch@ (Oct 10, 2013)

Something doesn't look right there. That doesn't look like a complete FTP connection negotiation. central.dogma should be responding back from port 21 to the random high port on gadget.dogma with the ports they will use before a passive ftp connection with random high to random high starts.  Are the central.dogma and gadget.dogma hosts in the same layer 2 network or are there more routers/firewalls involved?  Can you provide more network details like an `ifconfig` and `netstat -nr` on both ends if possible?


----------

