# What is the right approach to build ports with i386 CPU features on a 64bit machine?



## goshanecr (Jun 12, 2019)

Good day!

I have a far placed (in a village), old PC based on a Pentium-III. And I want to generate a simple FreeBSD based system on it.
How I do that:
1. On my home PC (Ryzen based), I'm start Virtualbox instance with FreeBSD 12-RELEASE i386.
2. I add *CPUTYPE?=pentium3* on /etc/make.conf
3. I rebuild system from sources (with that CPUTYPE), also install all needed software from ports.
4. Do a *dump *of generated system and *restore* on a real P3 based PC.
5. System works, but some software not starts on *P3* with error _"illegal instruction". _On a virtualbox instance that software works.

So I don't understand why, only one mention that clang may check current CPU capabilities and generate code with, for example SSE2, SSE3 etc. instructions which exists on a Ryzen CPU.
How can I solve that?


----------



## Phishfry (Jun 12, 2019)

If you look at /usr/src/share/mk/bsd.cpu.mk the proper target *CPUTYPE=p3* for desktop. There is a p3 mobile setting too.


----------



## goshanecr (Jun 12, 2019)

Phishfry said:


> the proper target *CPUTYPE=p3* for desktop. There is a p3 mobile setting too.


???
But... In /usr/share/examples/etc/make.conf

```
# Currently the following CPU types are recognized:
#   Intel x86 architecture:
#       (AMD CPUs)      amdfam10, opteron-sse3, athlon64-sse3, k8-sse3,
#                       opteron, athlon64, athlon-fx, k8, athlon-mp,
#                       athlon-xp, athlon-4, athlon-tbird, athlon, k7,
#                       geode, k6-3, k6-2, k6
#       (Intel CPUs)    core2, core, nocona, pentium4m, pentium4, prescott,
#                       pentium3m, pentium3, pentium-m, pentium2,
#                       pentiumpro, pentium-mmx, pentium, i486
#       (VIA CPUs)      c7, c3-2, c3
#   AMD64 architecture: amdfam10, opteron-sse3, athlon64-sse3, k8-sse3,
#                       opteron, athlon64, k8, core2, nocona
#   SPARC-V9 architecture:      v9 (generic 64-bit V9), ultrasparc (default
#                               if omitted), ultrasparc3
# Additionally the following CPU types are recognized by clang:
#   Intel x86 architecture (for both amd64 and i386):
#       (AMD CPUs)      znver1, bdver4, bdver3, bdver2, bdver1, btver2, btver1
#       (Intel CPUs)    cascadelake, tremont, goldmont-plus, icelake-server,
#                       icelake-client, cannonlake, knm, skylake-avx512, knl,
#                       goldmont, skylake, broadwell, haswell, ivybridge,
#                       sandybridge, westmere, nehalem, silvermont, bonnell
#
# (?= allows to buildworld for a different CPUTYPE.)
#
#CPUTYPE?=pentium3
```
And there is no *p3*, but I wil try.


----------



## k.jacker (Jun 12, 2019)

Do you want to prepare a harddisk for said P3 system?
I'd simply take an old IDE hard disk, create MBR table on it, freebsd-boot and freebsd-ufs partition and install bootcode.
Then extract kernel.txz and base.txz for i383 onto the hard disk and feed it with at least the necessary config files to be able to boot.
That should work, even on AMD64.
There are howtos and threads on that topic, here on there forum.


----------



## goshanecr (Jun 12, 2019)

No... I check /usr/src/share/mk/bsd.cpu.mk

```
.   elif ${CPUTYPE} == "p3"
CPUTYPE = pentium3
```
This is a aliases, so if I use *p3 *really it will be convert to *pentium3*


----------



## goshanecr (Jun 12, 2019)

k.jacker said:


> Do you want to prepare a harddisk for said P3 system?
> I'd simply take an old IDE hard disk, create MBR table on it, freebsd-boot and freebsd-ufs partition and install bootcode.
> Then extract kernel.txz and base.txz for i383 onto the hard disk and feed it with at least the necessary config files to be able to boot.
> That should work, even on AMD64.
> There are howtos and threads on that topic, here on there forum.


No, I prepare harddisk from live CD of FreeBSD 12 i386. But generate system on that old PC will be very long time and difficult because there is no internet and I go to it sometimes. So I generate system on virtualbox and dump/restore it on real P3 PC.


----------



## goshanecr (Jun 12, 2019)

I have core dump file from real P3 PC. Can it help to understand what going on?


----------



## k.jacker (Jun 12, 2019)

I don't understand why you fiddle with the CPU type and you don't need neither a live CD nor VirtualBox.
A Pentium 3 is a regular 32Bit x86 CPU, that works without tweaking with FreeBSD for i386.

Would you care to stop for a moment to explain what you are doing, but rather explain what your goal is?


----------



## goshanecr (Jun 13, 2019)

k.jacker said:


> I don't understand why you fiddle with the CPU type and you don't need neither a live CD nor VirtualBox.
> A Pentium 3 is a regular 32Bit x86 CPU, that works without tweaking with FreeBSD for i386.
> 
> Would you care to stop for a moment to explain what you are doing, but rather explain what your goal is?


I think, that you don't understand because of my bad english 
Explaining of what I want to finally get: I need a workstation based on 12-RELEASE i386 with all needed software for me.

Why I don't want to generate system directly on P3, because it will be too longtime. And because I don't want to transfer it from village to home, and because at village have no internet.

What problem I have:
1. Base system installed fine and it works (on P3 and on VBox)
2. Not works (on P3 only) some 3dparty software installed from ports (otter-browser, lighttpd, mariadb). On Vbox they all works.
3. I mean problem may be because of when building from ports on VBox, CPU capabilities shows like host PC (SSE, SSE2, SSE3, SSE4 .. etc) and it builds using it. 


Why I don't want to install all software from packages? Just because I don't want.


----------



## k.jacker (Jun 13, 2019)

Point 3. is your problem. A quick google search confirmed that the host's CPU capabilities are passed over to the guest.
So building the ports with your Ryzen's instruction set will not work without either tweaking VirtualBox or make.

I know little about VirtualBox and I've never tried building ports, targeting a CPU with different/less capabilities.
To avoid the XY problem, you should change your thread title to something like:

"What is the right approach to build ports with i386 CPU features on a 64bit machine?"


----------



## goshanecr (Jun 13, 2019)

k.jacker said:


> So building the ports with your Ryzen's instruction set will not work without either tweaking VirtualBox or make.


But why setting CPUTYPE=pentium3 is not enough to use instruction set by compiler? At first time I think that maybe because I set more flexible variant CPUTYPE?=pentium3, now I change it to = and also check all ports options to various SIMD, SSE optimization knobs. Don't want to rebuild all of them on a real machine. It will be nightmare.


----------



## shkhln (Jun 13, 2019)

goshanecr said:


> But why setting CPUTYPE=pentium3 is not enough to use instruction set by compiler?




```
% grep CPUTYPE -r /usr/ports/
/usr/ports/shells/sparforte/files/new_configure.in:CPUTYPE="@ARCH@"
/usr/ports/shells/sparforte/files/new_configure.in: -e "s|CPUTYPE|${CPUTYPE}|g" \
/usr/ports/shells/sparforte/files/new_configure.in: -e "s|CPUTYPE|${CPUTYPE}|g" \
/usr/ports/lang/tcc/Makefile:CPUTYPE=    x86_64
/usr/ports/lang/tcc/Makefile:CPUTYPE=    i386
/usr/ports/lang/tcc/Makefile:    ${INSTALL_DATA} ${WRKSRC}/lib/${CPUTYPE}/*.o ${STAGEDIR}${PREFIX}/lib/tcc
/usr/ports/lang/mono/pkg-message:* Build Mono and gtk+ (x11-toolkits/gtk20) without CPUTYPE and with the
/usr/ports/multimedia/gmmlib/Makefile:        -DGMMLIB_MARCH:STRING="${CPUTYPE}"
/usr/ports/multimedia/libva-intel-media-driver/Makefile:        -DUFO_MARCH:STRING="${CPUTYPE}"
/usr/ports/math/sfft/Makefile:IGNORE=    this port requires SSE2, and benefits from SSE3 -- set CPUTYPE\
/usr/ports/astro/boinc-astropulse/pkg-descr:The port automatically optimizes for the host CPU using the CPUTYPE
/usr/ports/astro/boinc-setiathome/Makefile:.if defined(CPUTYPE)
/usr/ports/astro/boinc-setiathome/Makefile:CFLAGS+=    -march=${CPUTYPE}
/usr/ports/astro/boinc-setiathome/pkg-descr:The port automatically optimizes for the host CPU using the CPUTYPE
/usr/ports/devel/llvm80/Makefile:# Armv6 and armv7 uses hard float abi, unless the CPUTYPE has soft in it.
/usr/ports/devel/llvm80/Makefile:.if ${ARCH:Marmv[67]*} && (!defined(CPUTYPE) || ${CPUTYPE:M*soft*} == "")
/usr/ports/devel/llvm70/Makefile:# Armv6 and armv7 uses hard float abi, unless the CPUTYPE has soft in it.
/usr/ports/devel/llvm70/Makefile:.if ${ARCH:Marmv[67]*} && (!defined(CPUTYPE) || ${CPUTYPE:M*soft*} == "")
/usr/ports/devel/llvm-devel/Makefile:# Armv6 and armv7 uses hard float abi, unless the CPUTYPE has soft in it.
/usr/ports/devel/llvm-devel/Makefile:.if ${ARCH:Marmv[67]*} && (!defined(CPUTYPE) || ${CPUTYPE:M*soft*} == "")
/usr/ports/devel/psptoolchain-newlib/Makefile:# psp-gcc doesn't need -march=... in CFLAGS (e.g. set by CPUTYPE in make.conf); also need -g on psp for some reason
/usr/ports/devel/psptoolchain-pspsdk-stage2/Makefile:# psp-gcc doesn't need -march=... in CFLAGS (e.g. set by CPUTYPE in make.conf)
/usr/ports/devel/llvm60/Makefile:# Armv6 and armv7 uses hard float abi, unless the CPUTYPE has soft in it.
/usr/ports/devel/llvm60/Makefile:.if ${ARCH:Marmv[67]*} && (!defined(CPUTYPE) || ${CPUTYPE:M*soft*} == "")
/usr/ports/devel/llvm40/Makefile:# Armv6 and armv7 uses hard float abi, unless the CPUTYPE has soft in it.
/usr/ports/devel/llvm40/Makefile:.if ${ARCH:Marmv[67]*} && (!defined(CPUTYPE) || ${CPUTYPE:M*soft*} == "")
```

Despite some maintainer confusion, as far as FreeBSD Ports are concerned this flag doesn't really exist.


----------



## goshanecr (Jun 13, 2019)

shkhln said:


> Despite some maintainer confusion, as far as FreeBSD Ports are concerned this flag doesn't really exist.


Maybe, but when port builds, it uses option CPUTYPE from /etc/make.conf as a _-march_ argument for CC.
Maybe there is a argument which I can add to CFLAGS which instruct clang to build code exactly for given CPU?


----------



## malavon (Jun 13, 2019)

If you don't want to use packages, that's fine whatever your particular reasons are. Look up on how to set up poudriere with a i386 jail. It's specifically developed to support this.
`pkg install poudriere`
Then see poudriere(8) or check out any of the tutorials found online or on this forum.

With poudriere you can build all packages you want, with all port options you want for i386 on a faster amd64 machine.
If you need to set any extra compilation options (note: no need in your case), you can modify any of the make.conf files that poudriere uses (from the man page):

```
/usr/local/etc/poudriere.d/make.conf
           /usr/local/etc/poudriere.d/<setname>-make.conf
           /usr/local/etc/poudriere.d/<tree>-make.conf
           /usr/local/etc/poudriere.d/<jailname>-make.conf
           /usr/local/etc/poudriere.d/<tree>-<setname>-make.conf
           /usr/local/etc/poudriere.d/<jailname>-<tree>-make.conf
           /usr/local/etc/poudriere.d/<jailname>-<setname>-make.conf
           /usr/local/etc/poudriere.d/<jailname>-<tree>-<setname>-make.conf
           /usr/local/etc/poudriere.d/hooks/plugins/<plugin>/make.conf
```

If you want to set or unset options for all packages (when you execute `poudriere options`), you can add

```
OPTIONS_UNSET+=SSE
```
 to one of the above make.conf files.
I don't have my builder online right now to check, but if I'm correct you can add multiple options separated by spaces.


----------



## SirDice (Jun 13, 2019)

malavon said:


> I don't have my builder online right now to check, but if I'm correct you can add multiple options separated by spaces.


Yep. This is fine for example:

```
OPTIONS_UNSET+= SSE X11
```
You can also get creative as the make.conf files are 'stacked'. For example, I have two ports trees; desktop and server. I have set several "overall" defaults in top level make.conf and have set `OPTIONS_UNSET+= X11` in server-make.conf. So for my 'server' based repositories you get a combination of make.conf and server-make.conf while my 'desktop' based repositories only use make.conf.


----------



## malavon (Jun 13, 2019)

SirDice said:


> For example, I have two ports trees; desktop and server.


Might be interesting to compare setups here. Before sets were added to poudriere, I used two jails (desktop and server) but I guess two ports trees were perfectly feasible as well for the same scenario.
Nowadays I use a single ports tree (default), a single jail (per architecture) but use a set each for desktop and server.
No need to duplicate anything anymore, poudriere handles this really nicely by separating port options. Two package files only and adding it to the poudriere command with `-z <set>`


----------



## SirDice (Jun 13, 2019)

malavon said:


> I used two jails (desktop and server) but I guess two ports trees were perfectly feasible as well for the same scenario.


I had that too. It was the naming convention of the resulting repository that prompted me to try with two ports trees. I liked the "jail name - ports name", for example 112-release-desktop or 120-release-server. Instead of ending up with 120-server-default. Two ports trees also makes some sense as servers have a different usage than desktop so I can update those differently.


----------



## malavon (Jun 13, 2019)

SirDice said:


> Instead of ending up with 120-server-default.


Fair enough. Note that sets end up with the default in reverse order (the set is at the back).
In my case that results in

```
url: "pkg+http://packages.malavon.com/${ABI}-default-desktop",
```
I kinda like the "default" part in that case, because it is my default desktop configuration 

Note to the OP: this isn't really relevant to you for now. Just go with a default poudriere setup until you have more experience with it.


----------



## goshanecr (Jun 13, 2019)

Guys, many thanks!
That thread are very informative for me.


----------



## PMc (Jun 14, 2019)

I am doing this with chroot. More or less something like:

```
chroot $bin_mp sh -c "cd /usr/src && \
              make -j$ncpu buildworld ${mach:+TARGET=$mach} \
            ${arch:+TARGET_ARCH=$arch} && \
        make installworld DESTDIR=/mnt \
            ${mach:+TARGET=$mach} ${arch:+TARGET_ARCH=$arch} && \
        make distribution DESTDIR=/mnt \
            ${mach:+TARGET=$mach} ${arch:+TARGET_ARCH=$arch} && \
        yes | make delete-old; \
        echo"' $? > /tmp/_result' >> $Log 2>&1
```

Then what is created in the /chroot/mnt is a new clean installation. One can then mount that somewhere appropriate, chroot into it, adjust the /etc/make.conf as desired, rebuild again (this time including the kernel), over-install it again and then start building ports within it.
For an pentium3 there should be TARGET and TARGET_ARCH set to "i386", and that should do. One can also set "CPUTYPE=p3", but that should not be necessary.

After that, I do transfer the /usr/src and /usr/obj to the target machine, and install from these as usual. The ports get packaged and then installed via pkg.


----------

