# network woes: arp: ww.xx.yy.zz moved from ...



## Chris_H (Jun 24, 2018)

Greetings,
A couple yrs ago I switched to using a cable modem for my upstream to the internet at home. My home
net consists of ~6-8 computers behind a 16 port unmanaged gigabit switch (tp-link TL-SG1016D). I got
a block of 9 static IP's from the cable co. They're all assigned to the comps behind the switch, and 1 to
the cable modem. But I'm always seeing the following in /var/messages
ww.xx.yy.zz moved from xx:xx:xx:xx:xx:xx to yy:yy:yy:yy:yy:yy on <nic driver name><device number>
I might expect this from a router; given the IP's are _technically_ assigned to the cable modem. But
not from a switch.
What's wrong? Should I add* staticarp*? How can I better diagnose _why_?

Thanks!

--Chris


----------



## Deleted member 54719 (Jun 24, 2018)

I have some idea, but please post the exact messages (understanding that you want to mask off the IP numbers for security reasons)...and are you seeing the MAC reassign message on every station attached to the switch?  Does the /var/messages have the complete list of reassignments on each machine, or only the reassignment for the local NIC?

Keep in mind that in the unmanaged switch there really isn't any such thing as "behind the switch" as each port has equal standing in that configuration.


----------



## Chris_H (Jun 24, 2018)

tempest766 said:


> I have some idea, but please post the exact messages (understanding that you want to mask off the IP numbers for security reasons)...and are you seeing the MAC reassign message on every station attached to the switch?  Does the /var/messages have the complete list of reassignments on each machine, or only the reassignment for the local NIC?
> 
> Keep in mind that in the unmanaged switch there really isn't any such thing as "behind the switch" as each port has equal standing in that configuration.


Thanks tempest766 , for the reply!
I don't understand what you're asking? Yes. The MAC is being moved. In fact this repeats in
the other computers /var/messages, as well. All of the computers NIC's "move".
I don't recall seeing this with "managed" switches.
I can't tell for sure. But it appears that the switch is adobting their MAC. Because the only one of the
MAC's I don't recognize is reportedly "Casa Systems Inc.". So apparently TP-LINK is using their stuff.

Thanks again, tempest766


----------



## Bobi B. (Jun 24, 2018)

IIRC this might happen with Wake-on-LAN, where router adopts MAC addresses of devices going into sleep mode, later waking them up with a magic packet. Or, if those IPs are public, your ISP might want to account all traffic, even between devices behind your cable modem, when modem is acting as a bridge, not as a router (my ISP does that for ETTH).


----------



## tingo (Jun 24, 2018)

Another thing to look out for is machines with multiple NICs (example: both wired and wireless). If both NICs are active at the same time, you will see those messages on FreeBSD machines connected to the same LAN. And in that situation, those messages are harmless.


----------



## Deleted member 54719 (Jun 24, 2018)

OK.  I probably will fail miserably in explaining this...but...here goes.  Unmanaged switches have no concept of routing IP, whereas managed switches have "some" IP routing capabilities.  Ethernet (not IP) is a next hop topology.  to send a message on an ethernet the two stations learn about each others MAC addresses and send traffic directly to the MAC of the receiving station in the "next hop".  This is the purpose of ARP.  What you are seeing is expected.  Each port on the switch also has a MAC address and the switch has a cache of MACs that are connected to the ports.  What is going on is essentially the propogation of that MAC cache to the individual stations since they cannot directly see the MAC on another port of the switch.  If it works, ignore it.  It is expected behaviour for your configuration.

Let me rephrase this a little better.  Without seeing your configuration I cannot be sure, but I'm guessing that what is happening is that the individual stations are changing their own MAC to match the MAC of the switch port to which they are plugged in.  This allow the station on the other end in the next hop to have a direct address for each connected machine.


----------



## Chris_H (Jun 24, 2018)

tempest766 said:


> OK.  I probably will fail miserably in explaining this...but...here goes.  Unmanaged switches have no concept of routing IP, whereas managed switches have "some" IP routing capabilities.  Ethernet (not IP) is a next hop topology.  to send a message on an ethernet the two stations learn about each others MAC addresses and send traffic directly to the MAC of the receiving station in the "next hop".  This is the purpose of ARP.  What you are seeing is expected.  Each port on the switch also has a MAC address and the switch has a cache of MACs that are connected to the ports.  What is going on is essentially the propogation of that MAC cache to the individual stations since they cannot directly see the MAC on another port of the switch.  If it works, ignore it.  It is expected behaviour for your configuration.
> 
> Let me rephrase this a little better.  Without seeing your configuration I cannot be sure, but I'm guessing that what is happening is that the individual stations are changing their own MAC to match the MAC of the switch port to which they are plugged in.  This allow the station on the other end in the next hop to have a direct address for each connected machine.


Thanks for the reply tempest766 !


tempest766 said:


> OK.  I probably will fail miserably in explaining this...but...here goes.


On the contrary; you did an excellent job. 
...and fairly well concurs with my current understanding. I only became concerned; in that I hadn't experienced these "messages" previously, and even then; I wasn't terribly concerned, in that the network just seemed to work. I'm only now investigating. Because I'm experiencing some "anomalies"; in that my email from the many FreeBSD lists I'm subscribed to, are coming in irradically. So I thought it time to get a _conclusive_ reason for the messages.
If it matters:

```
arp: ww.xx.yy.bb moved from 00:13:3b:0f:13:45 to 00:17:10:8e:0d:07 on re0
arp: ww.xx.yy.dd moved from 00:30:67:5e:29:5d to 00:17:10:8e:0d:07 on re0
arp: ww.xx.yy.ee moved from 00:17:10:8e:0d:07 to 00:24:81:ce:ba:87 on re0
arp: ww.xx.yy.cc moved from 00:17:10:8e:0d:07 to 00:13:3b:0f:13:44 on re0
arp: ww.xx.yy.ee moved from 00:17:10:8e:0d:07 to 00:24:81:ce:ba:87 on re0
arp: ww.xx.yy.cc moved from 00:17:10:8e:0d:07 to 00:13:3b:0f:13:44 on re0
arp: ww.xx.yy.bb moved from 00:13:3b:0f:13:45 to 00:17:10:8e:0d:07 on re0
arp: ww.xx.yy.dd moved from 00:17:10:8e:0d:07 to 00:30:67:5e:29:5d on re0
arp: ww.xx.yy.cc moved from 00:13:3b:0f:13:44 to 00:17:10:8e:0d:07 on re0
```
where re0 would be ww.xx.yy.aa
and where the ww.xx.yy range is from aa-ii
TOPOLOGY:

```
port #1 ==> cablemodem
port #2-9 ==> each of the CPU's
```
while pf(4) runs on all of the CPU's (computers), with the exception of one. They
all only have blocking tables. The other one, is used for (build) jails, and has the additional
entry to accommodate having cloned lo1 to permit the downloading of the src, and ports trees.
I should add, that the messages still occurred even prior to the cloning of lo1.

Thanks again, tempest766 !

--Chris


----------



## VladiBG (Jun 24, 2018)

Do you have access to the configuration of your router (cable modem)?


----------



## Chris_H (Jun 25, 2018)

Hello VladiBG , and thanks for the reply!
It's a docsys 3 based modem, and as a result, the "access" to much of the configuration is
fairly limited. I seem to remember having a web based GUI "setup", and possibly telnet(1) access
when I initially set it up.
Why do you ask?

Thanks again, VladiBG !
--Chris


----------



## VladiBG (Jun 25, 2018)

I'm asking you because your Casa System modem behaves strangely. Do you know how your IP address block is routed? Normally the network mask is either 255.255.255.240 for providing of 14 hosts or 255.255.255.248 for 6 hosts. That's why i'm curious how your router is configured to provide 9 hosts and what is your network mask on your computers.

Can you show how your subnet is configured and what is the default gateway that you are using (you can hide the first 3 octets of your IP addresses).

for example:

*subnet:* 192.168.1.0/28
*netmask:* 255.255.255.240
*hosts:* 192.168.1.1 to 192.168.1.14
*broadcast:* 192.168.1.15


----------



## Chris_H (Jun 25, 2018)

Well. It's _supposed_ to be:

```
ww.xx.yy.a1
ww.xx.yy.a2
ww.xx.yy.a3
ww.xx.yy.a4
ww.xx.yy.a5
ww.xx.yy.a6
ww.xx.yy.a7
ww.xx.yy.a8
GATEWAY:
ww.xx.yy.1
```
and _yes_ I guess that'd be 8 (assignable) IP's. I guess my mind added the GW. Looking
closer, just now, revealed 8+GW=9 
Thanks, VladiBG !
--Chris


----------



## VladiBG (Jun 25, 2018)

And what is your netmask?


----------



## Chris_H (Jun 25, 2018)

*DO'H*! 

That'd be 255.255.255.0


----------



## VladiBG (Jun 25, 2018)

So you are in the same network with another 254 hosts. Are you the only one connected behind the cable modem?
Is your IP address range public or private?


----------



## Chris_H (Jun 25, 2018)

Thanks for the reply, VladiBG !


VladiBG said:


> So you are in the same network with another 254 hosts.


Yes.


VladiBG said:


> Are you the only one connected behind the cable modem?
> Is your IP address range public or private?


Yep. I might add, that I'm also the *only* one on the /24, as well.

Thanks again, VladiBG !

--Chris


----------



## VladiBG (Jun 25, 2018)

Are you using DHCP? If some computer is changing his IP address and overlap with another computer with static address it may show you this error. Usually this shouldn't happen because the DHCP server fist check if the address is free.


----------



## Chris_H (Jun 25, 2018)

VladiBG said:


> Are you using DHCP? If some computer is changing his IP address and overlap with another computer with static address it may show you this error. Usually this shouldn't happen because the DHCP server fist check if the address is free.


I'm leasing a (contiguous) block of static IP's. So DHCP shouldn't figure into the equation.

Thanks for the reply, VladiBG !

--Chris


----------



## VladiBG (Jun 25, 2018)

Well the only possible ways that you are receiving this warning messages regarding arp moving are:
1. Overlapping two DHCP  servers in the same network.
2. Overlapping a static IP address with DHCP pool
3. Computer with the two NICs with the same IP address on both NICs (like a WiFi and Ethernet with the same IP)
4. You have two devices with the same IP address
5. Arp poisoning attack from bad user

The way that you can future investigate the problem is to check the MAC address of your device and verify which two are conflicting. You can do this by using Mac Address Lookup to identify the vendor of the device to help you to identify the device that is making the problem.


----------



## Chris_H (Jun 25, 2018)

Thatks, VladiBG , for your diligence on this!


VladiBG said:


> Well the only possible ways that you are receiving this warning messages regarding arp moving are:
> 1. Overlapping two DHCP  servers in the same network.
> 2. Overlapping a static IP address with DHCP pool
> 3. Computer with the two NICs with the same IP address on both NICs (like a WiFi and Ethernet with the same IP)
> ...





VladiBG said:


> The way that you can future investigate the problem is to check the MAC address of your device and verify which two are conflicting. You can do this by using Mac Address Lookup to identify the vendor of the device to help you to identify the device that is making the problem.


I used this method. I've checked all the NET related entries in rc.conf(5)  on all of the machines, as well as all of the DNS zones, and (DNS) configurations. But there is no overlap. As to (bad) user ARP poisoning; tho a valid point, is _highly_ unlikely. Given that I encountered the ARP announcements almost immediately. So I'm left, I think, with a possible hardware problem. I'm going to take a closer look at a HUB (wireless/4port) I have plugged into one of the ports on the switch, as a potential candidate. While it _does_ run DHCP for the clients connected to it. It has a _static_ IP, and the DHCP is for _internal_ IP's within it. The only other possibility I can conceive; is a 2 port PCIe NIC I have in one of the boxes ( re(4) ). Which has 2 static IP's attached to it. FreeBSD supports the card well, and dmesg(8), and ifconfig(8) indicate the card is working (behaving) as intended.

Thanks a _million_, VladiBG , for all your valuable input!

--Chris


----------

