# Jails not using multiple IPs



## oXoJBHC (Jul 15, 2013)

I cannot get jails to function with IPv6 addresses. I've been trying to get each jail an IPv4 and IPv6 address, but to simplify this issue I've created a test jail as IPv6 only.

I'm running 9.1-RELEASE-p3

My IPv6 addresses and domain have been sanitized, but it is a global prefix given by my ISP via DHCPv6 with prefix delegation (all handled by my router which is not this host). From the host I have full IPv6 Internet connectivity. 

IPv6 connectivity works fine from the host. I have security.jail.allow_raw_sockets set to 1, so ping should be allowed in jail for IPv4 and IPv6, and it works in IPv4.


```
# ping6 -c2 2001:4860:4860::8888
PING6(56=40+8+8 bytes) 2601:xxxx:yyyy:zzzz:215:17ff:fe0b:1386 --> 2001:4860:4860::8888
16 bytes from 2001:4860:4860::8888, icmp_seq=0 hlim=50 time=54.307 ms
16 bytes from 2001:4860:4860::8888, icmp_seq=1 hlim=50 time=69.797 ms

--- 2001:4860:4860::8888 ping6 statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 54.307/62.052/69.797/7.745 ms
```

Here's a couple of my jails:

```
# jls -v
   JID  Hostname                      Path
        Name                          State
        CPUSetID
        IP Address(es)
     3  podrick.my.domain.com /mnt/tank/jail/podrick
        3                             ACTIVE
        4
        192.168.10.35
        2601:xxxx:yyyy:zzzz::35
     7  test.my.domain.com  /mnt/tank/jail/test
        7                             ACTIVE
        5
        2601:xxxx:yyyy:zzzz::38
```
IPv4 to and from jails works fine:

```
# ping -c2 192.168.10.35
PING 192.168.10.35 (192.168.10.35): 56 data bytes
64 bytes from 192.168.10.35: icmp_seq=0 ttl=64 time=0.022 ms
64 bytes from 192.168.10.35: icmp_seq=1 ttl=64 time=0.032 ms

--- 192.168.10.35 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.022/0.027/0.032/0.005 ms
# jexec 3 ping -c2 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=0 ttl=45 time=40.076 ms
64 bytes from 8.8.8.8: icmp_seq=1 ttl=45 time=40.038 ms

--- 8.8.8.8 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 40.038/40.057/40.076/0.019 ms
```

But not IPv6:

```
# ping6 -c2 2601:xxxx:yyyy:zzzz::38
PING6(56=40+8+8 bytes) 2601:xxxx:yyyy:zzzz:215:17ff:fe0b:1386 --> 2601:xxxx:yyyy:zzzz::38

--- 2601:xxxx:yyyy:zzzz::38 ping6 statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
# jexec 7 ping6 -c2 2001:4860:4860::8888
ping6: UDP connect: Can't assign requested address
```


----------



## SirDice (Jul 15, 2013)

Please post the relevant rc.conf entries for the jail.


----------



## oXoJBHC (Jul 15, 2013)

*rc.conf entries*

/etc/rc.conf.local

```
#############podrick#########
jail_podrick_rootdir="/mnt/tank/jail/podrick/"
jail_podrick_hostname="podrick.my.domain.com"
jail_podrick_interface="lagg0"
jail_podrick_ip="192.168.10.35/32,2601:xxxx:yyyy:zzzz::35/128"
jail_podrick_exec_start=""
jail_podrick_exec_afterstart0=""
jail_podrick_exec_afterstart1=""
jail_podrick_exec_stop="/bin/sh /etc/rc.shutdown; /usr/local/etc/rc.d/plexmediaserver stop"
jail_podrick_flags=""
jail_podrick_fstab="/etc/fstab.podrick"
jail_podrick_devfs_ruleset=""
jail_podrick_mount_enable="YES"
jail_podrick_devfs_enable="YES"
jail_podrick_procfs_enable="NO"
jail_podrick_fdescfs_enable="NO"
#############test#########
jail_test_rootdir="/mnt/tank/jail/test/"
jail_test_hostname="test.my.domain.com"
jail_test_interface="lagg0"
jail_test_ip="2601:xxxx:yyyy:zzzz::38/128"
jail_test_exec_start=""
jail_test_exec_afterstart0=""
jail_test_exec_afterstart1=""
jail_test_exec_stop="/bin/sh /etc/rc.shutdown"
jail_test_flags=""
jail_test_fstab="/etc/fstab.test"
jail_test_devfs_ruleset=""
jail_test_mount_enable="YES"
jail_test_devfs_enable="YES"
jail_test_procfs_enable="NO"
jail_test_fdescfs_enable="NO"
```


----------



## oXoJBHC (Jul 15, 2013)

Also an `ifconfig`

```
# ifconfig lagg0
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
        ether 00:15:17:0b:13:86
        inet6 fe80::215:17ff:fe0b:1386%lagg0 prefixlen 64 scopeid 0x11
        inet 192.168.10.20 netmask 0xffffff00 broadcast 192.168.10.255
        inet6 2601:xxxx:yyyy:zzzz:215:17ff:fe0b:1386 prefixlen 64 autoconf
        inet 192.168.10.35 netmask 0xffffffff broadcast 192.168.10.35
        inet6 2601:xxxx:yyyy:zzzz::35 prefixlen 128 duplicated
        inet6 2601:xxxx:yyyy:zzzz::38 prefixlen 128 duplicated
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: active
        laggproto lacp lagghash l2,l3,l4
        laggport: em1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        laggport: em0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
```


----------



## SirDice (Jul 15, 2013)

It looks good, I can even see the IPv6 addresses on the interface as aliases. It may be due to the lagg(4) interface, I only tried it with a 'real' interface. From within a jail can you see the correct IP addresses with ifconfig(8)?


----------



## oXoJBHC (Jul 15, 2013)

Yes I can:

```
# jls -v
   JID  Hostname                      Path
        Name                          State
        CPUSetID
        IP Address(es)
     6  podrick.my.domain.com /mnt/tank/jail/podrick
        6                             ACTIVE
        4
        192.168.10.35
        2601:xxxx:yyyy:zzzz::35
     9  test.my.domain.com  /mnt/tank/jail/test
        9                             ACTIVE
        5
        2601:xxxx:yyyy:zzzz::38
# jexec 9 ifconfig lagg0
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
        ether 00:15:17:0b:13:86
        inet6 2601:xxxx:yyyy:zzzz::38 prefixlen 128 duplicated
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: active
        laggproto lacp lagghash l2,l3,l4
        laggport: em1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        laggport: em0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
```

I did notice something strange, maybe not an issue, but looks strange

```
# jls
   JID  IP Address      Hostname                      Path
     6  192.168.10.35   podrick.my.domain.com /mnt/tank/jail/podrick
     9  -               test.my.domain.com  /mnt/tank/jail/test
```

`jls` without `-v` does not show the IPV6 address for test, even though it's the only address for that jail.


----------

