# Wish to migrate exist ordinary jail to VIMAGE infrastructure



## epopen (Feb 7, 2019)

Hi All.

My server upgrade to 12.0-Release and found news about VIMAGE into GENERAL config, meant VIMAGE ready for product environment.
So I wish to migrate exist ordinary jail to VIMAGE infrastructure too .
But procedure stopped at default gateway.

Current ordinary jail environment detail description.
My server using  VDSL PPPoE dial-up by net/mpd5 , and get *pusedo *static IPv4.
*Dynamic *IPv6 get by net/dhcp6 .

Network configuration
`ifconfig`

```
wan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8000a<TXCSUM,VLAN_MTU,LINKSTATE>
        ether 00:1e:68:c4:e1:9e
        inet6 2001:b011:a480:592f:21e:68ff:fec4:e19e prefixlen 64
        inet6 fe80::21e:68ff:fec4:e19e%wan0 prefixlen 64 scopeid 0x1
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 10.0.0.254 netmask 0xffffff00
        inet 10.0.0.2 netmask 0xffffffff
        inet 10.0.0.1 netmask 0xffffffff
        inet6 fe80::1%lo1 prefixlen 64 scopeid 0x5
        inet6 fd00::ffff:ffff:fffe prefixlen 96
        inet6 fd00::ffff:a00:2 prefixlen 128
        inet6 fd00::ffff:a00:1 prefixlen 128
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo
ng0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1460
        inet6 fe80::547:4b0e:4df6:1f7c%ng0 prefixlen 64 scopeid 0x6
        inet 122.117.86.253 --> 168.95.98.254 netmask 0xffffffff
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
```
Here:

ng0:  Virtual interface created by net/mpd5 (VDSL dial-up PPPoE), attached *pusedo *static IPv4.
lo1:  Virtual interface clone from lo0, attach all of jail.
wan0: _*"Physical interface to VDSL modem"*_,  attached _*dynamic*_ IPv6 address by net/dhcp6 and got from above interface 'ng0'.
10.0.0.2 / fd00::ffff:a00:2: HTTP server Jail's address.
10.0.0.1 / fd00::ffff:a00:1: DNS server Jail's address.
/etc/jail.conf for ordinary jail.

```
allow.nomount;
allow.noraw_sockets;
allow.noset_hostname;
allow.nosysvipc;
exec.clean;
exec.jail_user          = "root";
exec.start              += "/bin/sh /etc/rc";
exec.stop               = "/bin/sh /etc/rc.shutdown";
exec.system_user        = "root";
host.hostname           = "epopen.com";
interface               = "lo1";
mount.devfs;
path                    = "/usr/jail/${name}";
persist;                                      

domain {
    ip4.addr            = "10.0.0.1";
    ip6.addr            = "fd00::ffff:a00:1";
}

http {
    ip4.addr            = "10.0.0.2";
    ip6.addr            = "fd00::ffff:a00:2";
}
```

Generated netgraph 


After Googled tutorial, I plan use /usr/share/examples/jails/jng to create VIMAGE.
Beasuse server using net/mpd5 , both same use netgraph(4) subsystem.

Below is testing  /etc/jail.conf for VIMAGE jail "httpd".

```
allow.nomount;
allow.noraw_sockets;
allow.noset_hostname;
allow.nosysvipc;
exec.clean;
exec.jail_user          = "root";
exec.start              += "/bin/sh /etc/rc";
exec.stop               = "/bin/sh /etc/rc.shutdown";
exec.system_user        = "root";
host.hostname           = "epopen.com";
mount.devfs;
path                    = "/usr/jail/${name}";
persist;                                      

domain {
    interface           = "lo1";
    ip4.addr            = "10.0.0.1";
    ip6.addr            = "fd00::ffff:a00:1";
}

http {
    vnet;
    vnet.interface      = "ng0_${name}";
    exec.prestart       += "/usr/share/examples/jails/jng bridge ${name} wan0";
    exec.poststop       += "/usr/share/examples/jails/jng shutdown ${name}";
}
```

Start jail by `service jail start httpd`
And check interface by `ifconfig` in jail.

```
lo0: flags=8008<LOOPBACK,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo
ng0_httpd: flags=8842<BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=28<VLAN_MTU,JUMBO_MTU>
        ether 02:c0:c5:c4:e1:9e
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
```
Confirm new interface _*ng0_httpd*_ generated.
And netgraph  can find new both _*wan0bridge *_and _*ng0_httpd*_ generated.

Next step, configurate default gateway.
I known all of googled tutorial's upstream interface is static IP.
But my server use  *pusedo *static IPv4.
I have no idea configre it and stopped here.

Can help me if possible?
Thanks all very much.


----------



## mickey (Feb 9, 2019)

First off, I don't know much about jails, so I'm not sure how much help I can provide regarding that matter. I have been using DSL PPPoE dialup for years though (i use ppp(8) from base system) and I can't help it, but something with your interface configuration looks off to me.

You say wan0 is a physical interface connected to your VDSL modem. Then the only thing ever using that interface should be net/mpd5 for sending/receiving PPPoE encapsulated frames from/to your modem. Unless you want to be able to access the modem's web/telnet based configuration, this interface probably doesn't even need an IP address assigned. For all other intents and purposes your 'internet interface' is the virtual interface created by net/mpd5.
Your public IPv4 and IPv6 addresses reside on different interfaces, which I think is wrong. The IPv4 address is automatically negotiated by net/mpd5 using PPP/IPCP and correctly gets assigned to your ng0 interface. PPP should also negotiate a link-local IPv6 address using IPV6CP and assign it to the ng0 interface. Your public IPv6 address can either be negotiated via SLAAC (for this to work you need to set 'ipv6_cpe_wanif=ng0' in /etc/rc.conf - see rc.conf(5)) and/or using `dhcp6c` (in this case `dhcp6c` should assign the IPv6 address to the ng0 interface, not wan0 - check your /usr/local/etc/dhcp6c.conf).
Your default route for IPv4 and IPv6 should automatically get set up by net/mpd5 whenever needed, unless you specifically configured it not to.
In your jail.conf (exec.prestart) you use interface wan0, which doesn't seem right (See 1.)


----------



## epopen (Feb 12, 2019)

mickey said:


> First off, I don't know much about jails, so I'm not sure how much help I can provide regarding that matter. I have been using DSL PPPoE dialup for years though (i use ppp(8) from base system) and I can't help it, but something with your interface configuration looks off to me.
> 
> You say wan0 is a physical interface connected to your VDSL modem. Then the only thing ever using that interface should be net/mpd5 for sending/receiving PPPoE encapsulated frames from/to your modem. Unless you want to be able to access the modem's web/telnet based configuration, this interface probably doesn't even need an IP address assigned. For all other intents and purposes your 'internet interface' is the virtual interface created by net/mpd5.
> Your public IPv4 and IPv6 addresses reside on different interfaces, which I think is wrong. The IPv4 address is automatically negotiated by net/mpd5 using PPP/IPCP and correctly gets assigned to your ng0 interface. PPP should also negotiate a link-local IPv6 address using IPV6CP and assign it to the ng0 interface. Your public IPv6 address can either be negotiated via SLAAC (for this to work you need to set 'ipv6_cpe_wanif=ng0' in /etc/rc.conf - see rc.conf(5)) and/or using `dhcp6c` (in this case `dhcp6c` should assign the IPv6 address to the ng0 interface, not wan0 - check your /usr/local/etc/dhcp6c.conf).
> ...


Hi mickey
Thanks your post 
About public IPv4 and IPv6 addresses reside on different interface, because I could get it from ISP by net/dhcp6.
But `dhcp6c` couldn't assign it to ng0 interface, wan0 could.

IPv6 addresses assigned to ng0 interface after your suggested 'ipv6_cpe_wanif=ng0' added.
Thanks you very much.


----------



## mickey (Feb 12, 2019)

epopen said:


> About public IPv4 and IPv6 addresses reside on different interface, because I could get it from ISP by net/dhcp6.
> But `dhcp6c` couldn't assign it to ng0 interface, wan0 could.
> 
> IPv6 addresses assigned to ng0 interface after your suggested 'ipv6_cpe_wanif=ng0' added.



So your ng0 interface now gets an IPv6 address? Is it a different IPv6 prefix than the one assigned to wan0? What does the output of `ifconfig ng0` and `ifconfig wan0` and your /usr/local/etc/dhcp6c.conf look like now? I suspect that your ng0 interface now gets an IPv6 address via SLAAC from your ISPs router advertisments, and unless you need additional IPv6 prefixes (say for computers connected to your LAN/WLAN - which of course would imply that this machine is actually a router and not just a server) you don't even need to use dhcpv6 at all.

About your default routes, what does `netstat -rn` show?


----------



## epopen (Feb 13, 2019)

mickey said:


> So your ng0 interface now gets an IPv6 address? Is it a different IPv6 prefix than the one assigned to wan0? What does the output of `ifconfig ng0` and `ifconfig wan0` and your /usr/local/etc/dhcp6c.conf look like now? I suspect that your ng0 interface now gets an IPv6 address via SLAAC from your ISPs router advertisments, and unless you need additional IPv6 prefixes (say for computers connected to your LAN/WLAN - which of course would imply that this machine is actually a router and not just a server) you don't even need to use dhcpv6 at all.
> 
> About your default routes, what does `netstat -rn` show?


Hi mickey

Yes, ng0 interface get TWO IPv6 address, IPv6 prefix /48 same with wan0 interface as below.
`ifconfig` result.

```
wan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8000a<TXCSUM,VLAN_MTU,LINKSTATE>
        ether 00:1e:68:c4:e1:9e
        inet6 2001:b011:a480:5110:21e:68ff:fec4:e19e prefixlen 64
        inet6 fe80::21e:68ff:fec4:e19e%wan0 prefixlen 64 scopeid 0x1
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
........
ng0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1460
        inet6 fe80::c851:2c39:1b39:cb47%ng0 prefixlen 64 scopeid 0x6
        inet6 2001:b011:a480:a3d:c851:2c39:1b39:cb47 prefixlen 64 autoconf
        inet6 2001:b011:a480:a3d:a544:8459:ae77:933 prefixlen 64 autoconf temporary
        inet 122.117.86.253 --> 168.95.98.254 netmask 0xffffffff
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
```

And /usr/local/etc/dhcp6c.conf is.

```
interface ng0 {
        send ia-pd 0;
#       request domain-name-servers;
#       request domain-name;
        script "/usr/local/etc/dhcp6c-script";
};

id-assoc pd 0 {
        prefix-interface wan0 {
                sla-id 0;
                sla-len 0;
        };
};
```

Yesterday, I found ISP's official IPv6 configuration document.
Page 36 (D-Link CPE DIR-825) wrote.

Enable DHCP-PD
Autoconfiguration Type = SLAAC+Stateless DHCPv6
These accord with your explain 
About both *SLAAC* and _*Stateless DHCPv6*_, I asking ISP service center about what information provide from both.
DHCPv6 must be use look like.

And the document wrote Windows、Unix-based machine use "SLAAC + Stateless DHCPv6" also.

And route table by `netstat -r`

```
Internet6:
Destination        Gateway            Flags     Netif Expire
::/96              localhost          UGRS        lo0
default            fe80::d2f0:dbff:fe UGS         ng0
localhost          link#2             UH          lo0
::ffff:0.0.0.0/96  localhost          UGRS        lo0
2001-b011-a480-0a3 link#6             U           ng0
2001-b011-a480-0a3 link#6             UHS         lo0
2001-b011-a480-0a3 link#6             UHS         lo0
2001-b011-a480-511 link#1             U          wan0
2001-b011-a480-511 link#1             UHS         lo0
fd00::ffff:0:0/96  link#5             U           lo1
dns.epopen.com     link#5             UHS         lo0
www.epopen.com     link#5             UHS         lo0
host.epopen.com    link#5             UHS         lo0
fe80::/10          localhost          UGRS        lo0
fe80::%wan0/64     link#1             U          wan0
fe80::21e:68ff:fec link#1             UHS         lo0
fe80::%lo0/64      link#2             U           lo0
fe80::1%lo0        link#2             UHS         lo0
fe80::%lo1/64      link#5             U           lo1
fe80::1%lo1        link#5             UHS         lo0
fe80::%ng0/64      link#6             U           ng0
fe80::c851:2c39:1b link#6             UHS         lo0
ff02::/16          localhost          UGRS        lo0
```
IPv6 default gateway is *ng0.*

Addition question by *SLAAC *IPv6 update.
Prior. DNS server's IPv6 record update by dns/ddclient and trigger by net/dhcp6 via /usr/local/etc/dhcp6c-script when IPv6 REBIND/RENEW.
But SLAAC haven't similar mechanism I known.
I plan to check it modify and update periodic.
Have you better method?

Thanks you very much.


----------



## mickey (Feb 13, 2019)

epopen said:


> Yes, ng0 interface get TWO IPv6 address, IPv6 prefix /48 same with wan0 interface as below.
> `ifconfig` result.
> 
> ```
> ...


Prefix on ng0 is 2001:b011:a480:a3d/64. The reason why you get two IPv6 addresses is that you probably have enabled IPv6 privacy (which is OK) by setting `ipv6_privacy="YES"` in /etc/rc.conf. The IPv6 privacy address is not derived from your hardware MAC address, is marked as 'temporary' and will be be the preferred address on that interface, so no worries there. On ng0 you now got an IPv4 address, link-local IPv6 address, IPv6 address and IPv6 privacy address, that's just fine.

On your wan0 interface however you still get a public IPv6 address which shouldn't be happening. This address is assigned by dhcp6 prefix delegation (PD), which is essentially you telling your ISP: "Hey, I got a few comrades here in my local area network that need IPv6 addresses too, can you delegate me another prefix?". This of course is not needed if there is no local area network with other machines and this machine is the only one. In that case you should be using dhcp6 in 'information-only' mode, which will only request information about DNS name servers etc from your ISP's dhcp6 server, but *not* request any prefixes/addresses.



epopen said:


> And /usr/local/etc/dhcp6c.conf is.
> 
> ```
> interface ng0 {
> ...


That's exactly the culprit. Here you tell dhcp6c to request being delegated a prefix and assign that prefix to your wan0 interface. To have dhcp6c only request information about IPv6 DNS servers, the configuration should look something like this:

```
interface ng0 {
        request domain-name-servers;
        information-only;
        script "/usr/local/etc/dhcp6c-script";
};
```



epopen said:


> And route table by `netstat -r`
> 
> ```
> Internet6:
> ...


Looks good, that's exactly where the default route should be pointing to. I assume you got an IPv4 default route also, so your routing should be in order.



epopen said:


> Addition question by *SLAAC *IPv6 update.
> Prior. DNS server's IPv6 record update by dns/ddclient and trigger by net/dhcp6 via /usr/local/etc/dhcp6c-script when IPv6 REBIND/RENEW.
> But SLAAC haven't similar mechanism I known.
> I plan to check it modify and update periodic.
> Have you better method?


That should work almost the same as before as you will still be using dhcp6c to request information about IPv6 DNS servers from your ISP's dhcp6 server. When that happens, the script you defined in your /usr/local/etc/dhcp6c.conf will be invoked and can use the information supplied via environment variables to do whatever you see fit (update dynamic DNS, etc). The only thing that needs to be changed with your script is the invocation reason that it reacts to. Instead of REBIND/RENEW you will probably want to use INFOREQ. From dhcp6c(8):

```
Configuration Script
     When dhcp6c receives a reply message, it will invoke a supplementary
     configuration script specified in the dhcp6c.conf(5) file.  The daemon
     will provide the script with configuration parameters as environment
     variables, which include:

     REASON  The reason why the script is invoked.  One of the following
             values is set based on a message which the client sent before the
             REPLY message arrives:
                   INFOREQ  The client sent an INFORMATION REQUEST message.
                   REQUEST  The client sent a REQUEST message.
                   RENEW    The client sent a RENEW message.
                   REBIND   The client sent a REBIND message.
                   RELEASE  The client sent a RELEASE message.
                   EXIT     This happens when dhcp6c exits.
     new_domain_name_servers
             A list of available DNS servers, each of which is an IPv6 numeric
             address and is separated by a white space character.
     new_domain_name
             A list of DNS names, which provides DNS name search path.
     new_ntp_servers
             A list of available NTP servers, each of which is an IPv6 numeric
             address and is separated by a white space character.
     new_sip_servers
             A list of available SIP server addresses, each of which is an
             IPv6 numeric address and is separated by a white space character.
     new_sip_name
             A list of SIP server domain names.
     new_nis_servers
             A list of available NIS server addresses, each of which is an
             IPv6 numeric address and is separated by a white space character.
     new_nis_name
             A list of NIS domain names.
     new_nisp_servers
             A list of available NIS+ server addresses, each of which is an
             IPv6 numeric address and is separated by a white space character.
     new_nisp_name
             A list of NIS+ domain names.
     new_bcmcs_servers
             A list of available BCMCS server addresses, each of which is an
             IPv6 numeric address and is separated by a white space character.
     new_bcmcs_name
             A list of BCMCS server domain names.

     Note that the daemon does not always provide all the parameters.  It sets
     an environment variable only when the corresponding configuration
     parameter is provided by the DHCPv6 server.
```

Another method of getting the IPv6 DNS server information is via SLAAC using a feature called RDNSS, but I doubt that your ISP supports it, as it's use is still not very widespread. For this to work your ISP's router needs to include the IPv6 DNS server information within it's router advertisement messages. If you want to check, you could do something like `tcpdump -nvvvi ng0 icmp6` and wait until you see one of the periodic router advertisment messages sent by your ISP's router. This can take a couple of minutes depending on how often your ISP's router sends those, usually every 5-15 minutes. If it supports RDNSS then you should see an rdnss option included in the router advertisment specifying at least one IPv6 DNS server.

So if everything is working, you should have IPv4/IPv6 internet connectivity and something like `ping6 ipv6.google.com` and `traceroute6 ipv6.google.com` should work to verify IPv6 is working and the equivalent for IPv4: `ping ipv4.google.com` and `traceroute ipv4.google.com`


----------



## epopen (Feb 18, 2019)

Hi mickey

Sorry reply late.

RA message and DHCPv6 both provide IPv6, I asking ISP network customer service a lot of question, both usage included.
DHCPv6 attached IPv6 address removed. as below /usr/local/etc/dhcp6c.conf

```
interface ng0 {
        request domain-name-servers;
        information-only;
        script "/usr/local/etc/dhcp6c-script";
};
```
ISP provided domain-name-servers only confirmed.
ISP doesn't support RDNSS.

About your suggested `ipv6_privacy="YES"`, added into /etc/rc.conf
It  work fine, result below

```
ng0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1460
        inet6 fe80::21ce:dd50:1703:cd73%ng0 prefixlen 64 scopeid 0x6 
        inet6 2001:b011:a480:47e2:21ce:dd50:1703:cd73 prefixlen 64 autoconf 
        inet6 2001:b011:a480:47e2:1d0:82a3:51e4:ff7 prefixlen 64 deprecated autoconf temporary 
        inet6 2001:b011:a480:47e2:104d:a54e:c923:8a29 prefixlen 64 autoconf temporary 
        inet 122.117.86.253 --> 168.95.98.254 netmask 0xffffffff 
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
```
*deprecated* address rotate per 24 hours.

And `ping` result.

```
# ping6 ipv6.google.com
PING6(56=40+8+8 bytes) 2001:b011:a480:47e2:104d:a54e:c923:8a29 --> 2404:6800:4012:1::200e
16 bytes from 2404:6800:4012:1::200e, icmp_seq=0 hlim=53 time=14.528 ms
```
System using IPv6 address non-deprecated temporary address, correctly.

New system IPv6 configure.

Use IPv6 address from SLAAC RA packet, default gateway also.
Use dns/ddclient for DNS IPv6 record update per 300 seconds periodic from ''ng0' interface.
For security reason, use non-deprecated temporary address.
PS: 300 seconds OLD address windows is problem free, because incoming connection can be use *deprecated* address without problem.
Use net/dhcp6 for domain-name-servers config via `resolvconf` in  /usr/local/etc/dhcp6c-script
Thank you for all your assistance _I really appreciate your help in completed my IPv6 configure and much knowledge learn. _
Next step is migrate jail


----------



## mickey (Feb 22, 2019)

epopen said:


> Thank you for all your assistance _I really appreciate your help in completed my IPv6 configure and much knowledge learn. _
> Next step is migrate jail


Glad I could help.

I've been looking at your /etc/jail.conf and also at the guide you linked in your original post:


epopen said:


> Below is testing  /etc/jail.conf for VIMAGE jail "httpd".
> 
> ```
> allow.nomount;
> ...


I noticed two things about that configuration:

The http jail uses ng0_http as it's interface name. It's got the jail name appended, but still, you already got an ng0 interface, which is your internet connection. Just to avoid possible interferance I would rather use ng1 instead.
It uses wan0 as the bridge interface which cannot be correct as that interface is used for PPPoE exclusively. Nothing else should ever touch that interface.
So the configuration for your http jail could look somewhat like this:

```
http {
    vnet;
    vnet.interface      = "ng1_${name}";
    exec.prestart       += "/usr/share/examples/jails/jng bridge ${name} ng0";
    exec.poststop       += "/usr/share/examples/jails/jng shutdown ${name}";
}
```
This should create an ng1_http interface for your jail which is bridged to your internet interface ng0. If that works, try if you have connectivity from within your jail.


----------



## epopen (Feb 25, 2019)

Hi *mickey*

Thanks your replied.

I tried something and result below

```
# jng bridge httpd ng1
ifconfig: interface ng1 does not exist
```
Look like assign interface must be existence.

So back to use `jng` (implement with ng_bridge(4)) @ interface _*ng0*_

```
# jng bridge httpd ng0
ngctl: send msg: Function not implemented
```

Next use `jib` (implement with if_bridge(4)) @ interface _*ng0*_

```
# jib addm httpd ng0
ifconfig: BRDGADD ng0: Invalid argument
```

Both failure.
After traced and googled, found "Either  ng_bridge(4) or if_bridge(4)) both all of connect with interface created by ng_ether(4) required" result.
Unfortunately, interface *ng0 *created by ng_iface(4), so can't use.

I thinking about it.
I known create bridge with ng_bridge(4) and connect with jail's interface create by ng_ether(4) (Yes, these part of  `jng`).
But it isolate network,
I have no idea how connect the network to interface _*ng0*_ right now.

Thanks all.


----------



## mickey (Feb 25, 2019)

epopen said:


> Both failure.
> After traced and googled, found "Either  ng_bridge(4) or if_bridge(4)) both all of connect with interface created by ng_ether(4) required" result.
> Unfortunately, interface *ng0 *created by ng_iface(4), so can't use.


Is the ng_ether module loaded? According to /usr/share/examples/jails/README it has to be:

```
FreeBSD 12 has VIMAGE enabled in GENERIC on amd64.
In that case, for jng simply load the ng_ether module first
(ng_bridge and ng_eiface will load on demand):

        # (as root)
        # Load the ng_ether module at boot:
        $ sysrc kld_list+=ng_ether
        # Load ng_ether at once without rebooting:
        $ kldload ng_ether
```
Unfortunately the other information there is rather vague. If it comes down to an _ethernet_ interface being *required* as the base interface for the jails, the only solution is probably to assign a private IPv4 address to the wan0 interface (why is it called wan0 anyway?), something like 10.0.0.1/24 and try creating the jails using this interface instead. Unless you got another ethernet interface you did not mention. The ng0 interface is not an ethernet interface, it's point-to-point created by mpd5.


----------



## epopen (Feb 26, 2019)

mickey said:


> Is the ng_ether module loaded? According to /usr/share/examples/jails/README it has to be:
> 
> ```
> FreeBSD 12 has VIMAGE enabled in GENERIC on amd64.
> ...


Hi Mickey.

Yes, I had been loaded these in /boot/loader.confas below

```
ng_bridge_load="YES"
ng_eiface_load="YES"
if_epair_load="YES"
if_bridge_load="YES"
```

Thanks your good suggestion, I shall be think about detail


----------



## mickey (Feb 26, 2019)

epopen said:


> Yes, I had been loaded these in /boot/loader.confas below
> 
> ```
> ng_bridge_load="YES"
> ...


Ok, but I don't see ng_ether among these.


----------



## epopen (Feb 26, 2019)

mickey said:


> Ok, but I don't see ng_ether among these.


Ｈi Mickay

It build into kernel @ kernel configure file as below

```
options         NETGRAPH
options         NETGRAPH_ETHER
options         NETGRAPH_IFACE
options         NETGRAPH_PPP
options         NETGRAPH_PPPOE
options         NETGRAPH_SOCKET
options         NETGRAPH_TEE
```
Known net/mpd5 require that component so build into kernel directly 

Thanks you a lot.


----------

