# Linux-C7 get elf binary to run on other system



## SurtursRevenge (May 28, 2021)

Hello,

I hope this is the right section for my issue, but i was not sure if it belongs here or the system general.

I have a FreeBSD VM running with linux-c7 installed and configured. There I can run my linux elf binary without any issues.
I have now setup a new VM on a different system and did the same config. So I installed linux-c7 and did setup the needed mountpoints etc. I think I did the same as on the system where it is working.

The issue I am now facing is, that when I run the exact same linux elf binary on the fresh installed system it is hanging for about 2 minutes and than gives me a segmentation fault. While hanging the system console prints "syscall get_mempolicy not implemented".

So I am now asking my self where the difference between the system is and how I can get my binary to work.
I have triple checked every setting now and did not notice any difference. Also I tried to troubleshoot the "get_mempolicy" syscall error but was not able to find any solution.

I am now looking for any ideas on what I can also check so I get the binary to run on the new vm.


----------



## SirDice (May 28, 2021)

What version of FreeBSD? And do you have `linux_enable="YES"` in /etc/rc.conf?


----------



## SurtursRevenge (May 28, 2021)

The working vm. Is an outdated 12.1 if i remember correctly. 
For the new VM i tested 12.1 and 13.0.

linux_enable="YES" is set on both machines and i also verified that the linux kernel modules are loaded


----------



## SirDice (May 28, 2021)

The syscall is a kernel function and I would expect 13.0 to support the same or more than what linux(4) would support on 12.1. So I think that error might be a red herring. But maybe there's a difference in the configuration of the VM itself? 

Also check /etc/sysctl.conf and /boot/loader.conf. Look for `compat.linux.osrelease`, maybe that's been set to a specific version. Also verify if you have the same linux-c7-* ports/packages installed.


----------



## SurtursRevenge (May 29, 2021)

So i checked everything as asked. There is no specific version defined in sysctl or loader.conf
The working vm is running 12.1-RELEASE-p8
I also setup a new clean VM with 13.0 and checked everything again. All linux-c7 packages are installed the same

13.0 has a slightly newer version of "centos-release" 
But i still get "Segmentation fault (core dumped)" with a "syscall get_mempolicy not implemented"
Ofcourse i used brandelf for the binary before running it.


----------



## SirDice (May 29, 2021)

SurtursRevenge said:


> The working vm is running 12.1-RELEASE-p8
> I also setup a new clean VM with 13.0 and checked everything again. All linux-c7 packages are installed the same


That's not what I meant, your VM software sets up VM with a certain configuration, you should verify if that's the same, i.e. using the same virtual hardware settings.



SurtursRevenge said:


> 13.0 has a slightly newer version of "centos-release"


All versions of FreeBSD use the same ports tree. There is no version difference of ports/packages between versions of FreeBSD.


----------



## SurtursRevenge (May 29, 2021)

SirDice said:


> That's not what I meant, your VM software sets up VM with a certain configuration, you should verify if that's the same, i.e. using the same virtual hardware settings.


Sorry for the missleading answer. I just tought i give it a try with a fresh install as i maybe debugged the old one to death.
Both VMs run on VMware workstation.
The working one uses VM Version 8
The new one VM Version 12
So i dont think this should make any difference to use a more recent VM "Hardware" generation.



SirDice said:


> All versions of FreeBSD use the same ports tree. There is no version difference of ports/packages between versions of FreeBSD.


doing a cat /compat/linux/etc/centos-release gives me to different versions. 
On the working one i get 7.8.2003
on the new one i get 7.9.2009

But this could be my fault as i didnt patch the working one for a while


----------



## SirDice (May 29, 2021)

SurtursRevenge said:


> The working one uses VM Version 8
> The new one VM Version 12
> So i dont think this should make any difference to use a more recent VM "Hardware" generation.


It does.



SurtursRevenge said:


> doing a cat /compat/linux/etc/centos-release gives me to different versions.


Yes, that's just because you haven't updated your packages on that 12.1 machine (which you can't do any more because 12.1 is EoL).


----------



## SurtursRevenge (May 29, 2021)

SirDice said:


> It does.



So should i give it a try using the old VM generation hardware ? 
These VMs are at the moment my testing / staging env. In production this should be running primary on bare metal servers or sometimes on vmware esxi.
So i am trying to get this working "in general" 



SirDice said:


> Yes, that's just because you haven't updated your packages on that 12.1 machine (which you can't do any more because 12.1 is EoL).



Yes i know. I would first need to do an freebsd-update to a supported version and then update my linux-c7. I just forgot about updating as this machine was sitting on my pc and was not used some time ago.


----------



## SirDice (May 29, 2021)

I believe that VMWare hardware version difference is the key here. That `get_mempolicy()` is a kernel call for NUMA, I don't think that "old" hardware 8 version supports that. So it'll probably not get called at all.


----------



## SurtursRevenge (May 29, 2021)

So i now did a fresh install of freebsd 13.0 with a vm hardware version 8. It is still the same issue. Ich also checked that linux-c7-numactl-libs are installed, but this does not help.
The vm is now the exact same as the working one. 1 CPU Socket with 2 vCores on Vm version 8

So i was also wondering, because i tought numa is only required when having multiple CPU Sockets.


----------

