# How to compile a kernel-dependent library for i386 on x64 FreeBSD platform?



## battleship_potemkin (Mar 15, 2012)

Hi,

Thanks everyone for your help you were all provided by answering standard questions on this forum.
I seems like run into not-very standard issue and I would appreciate a help with this.

I'm developing an application polling kernel information like memory, cpu, disks load, etc and I want to be able to compile this library for bot i386 and x64 from OS x64 build.

Application builds just fine for i386 on i386, and for x64 on x64, but I want to keep one instance of FreeBSD and make cross-compilation for both platforms.

I made a search and tried a solution suggested at another thread http://forums.freebsd.org/showthread.php?t=14400, but it didn't help.

In case if compilation is done with fully custom include dirs:

```
CFLAGS += -m32 -march=i686 -mmmx -msse -msse2 -mfancy-math-387 -DCOMPAT_32BIT -L/usr/lib32 -B/usr/lib32 -I/usr/src/sys/i386/include -I/usr/src/sys -I. -nostdinc
```

Than the trouble is that compiler can't find files, that are included by standard headers:


```
In file included from sysinfo-bsd.c:1:
/usr/src/sys/sys/types.h:44:28: error: machine/endian.h: No such file or directory
In file included from /usr/src/sys/sys/types.h:45,
                 from sysinfo-bsd.c:1:
/usr/src/sys/sys/_types.h:33:28: error: machine/_types.h: No such file or directory
```


In case if header files paths are standard, the trouble is with C type's sizes:


```
In file included from /usr/include/machine/proc.h:36,
                 from /usr/include/sys/proc.h:66,
                 from sysinfo-bsd.c:14:
/usr/include/machine/segments.h:96: error: width of 'sd_hibase' exceeds its type
/usr/include/machine/segments.h:114: error: width of 'gd_hioffset' exceeds its type
/usr/include/machine/segments.h:184: error: width of 'rd_base' exceeds its type
*** Error code 1
```

It seems like kernel dependable application needs something more complicated than just a compiler flags - any help with finding a right way would be much appreciated.

Thanks.


----------



## SirDice (Mar 15, 2012)

Remove the CFLAGS, they'll probably cause more problems than solve them.

It should simply be a matter of setting TARGET:

```
make TARGET=i386 buildworld
make TARGET=i386 buildkernel
```


----------



## battleship_potemkin (Mar 15, 2012)

Thank you for the prompt reply.

Just to confirm - I'm not building a kernel, but an application/library that depends from the kernel system calls / headers - not sure I can use 'buildkernel' for this purpose?

TARGET is a variable handled inside FreeBSD source code right? I use my own Makefile for my application, it doesn't seems like it could be handled automatically without any actions from my side?

Please, correct me if I'm wrong.


----------



## SirDice (Mar 15, 2012)

Not sure if that also works for ports. ports-mgmt/tinderbox for example creates several chroot/jail environments and builds ports for different architectures inside them. You can run an i386 jail on amd64.

That's something I have done. Build a jail using TARGET=i386 and build i386 ports inside it. Most ports run fine inside it. But yours might have issues as you are using kernel structures. A i386 jail would still have a amd64 kernel, commands like ps(1) fail for this reason. It should be good enough to build though.


----------



## battleship_potemkin (Mar 16, 2012)

Thank you for the suggestion. I went through the documentation, but I didn't find how it could help...

But here is an idea (thanks!) - if speaking about jails - I need a complete installation of FreeBSD i386 and x64 and jail compilation of the tool there.

For example:

~/freebsd82-i386/ - complete FreeBSD 8.2 i386 installation including sources;
~/freebsd90-x64/ - complete FreeBSD 8.2 x64 installation including sources.

Then I should be able to compile my tool for FreeBSD 8.2 i386 in 'chroot ~/freebsd82-i386/' and for FreeBSD 9.0 x64 in 'chroot ~/freebsd90-x64/'.

If that sounds right - how can I have a complete installation of FreeBSD inside specific directory and keep it updated there?


----------



## SirDice (Mar 16, 2012)

You don't need separate sources, you can use the same source tree on both jails. The source tree is the same tree for all architectures.

But a basic jail is installed like so:

```
setenv D /jails/j-build-amd64
make -C /usr/src buildworld
make -C /usr/src installworld DESTDIR=${D}
make -C /usr/src/etc distribution DESTDIR=${D}
cd ${D} && ln -sf dev/null kernel
touch ${D}/etc/fstab
```

Add TARGET=i386 to the make commands to build an i386 jail.

This is just for convenience, in /etc/rc.conf:

```
jail_build_rootdir="/jails/j-build-amd64/"
jail_build_hostname="j-build-amd64.dicelan.home"
jail_build_flags="-l -U root -n build"
jail_build_ip="192.168.10.200,2001:xxxx:xxxx::200"
jail_build_interface="re0"
jail_build_mount_enable="YES"
jail_build_procfs_enable="YES"
jail_build_devfs_enable="YES"
jail_build_devfs_rules="jail"
```
Again, for convenience I also created a /etc/fstab.build:

```
/usr/src                        /jails/j-build-amd64/usr/src                    nullfs  ro,noatime      0       0
/usr/ports                      /jails/j-build-amd64/usr/ports                  nullfs  rw,noatime      0       0
```
This will mount the host's /usr/src/ and /usr/ports in the jail. So you don't need separate trees.

Start the jail with `# service jail start build` When it's started you can easily access it with `# jexec build su -`

I use this setup to create my own package repository.


----------



## battleship_potemkin (Mar 16, 2012)

Thank you. Once that is complete, I need to *chroot $D* and build my app?

Another question: can I, by using FreeBSD 9 x64, build in the same way FreeBSD 8.2 i386?  



> you can use the same source tree on both jails



That is not quite true, unfortunately.

The reason is that, for example, /usr/include/machine/segments.h on x64 is installed from the x64 platform, meaning no i386 application could be builtd, just because types mismatch (see errors in my first message). That is actually the main reason why I bother about a full valid installation and that's why I need the appropriate source directory from the root.


----------



## battleship_potemkin (Mar 16, 2012)

Apologize for the mistakes, my spell checker was switched to another language and I can't find no 'edit' option for this post


----------



## SirDice (Mar 16, 2012)

battleship_potemkin said:
			
		

> Thank you. Once that is complete, I need to *chroot $D* and build my app?


I'd do `# jexec <jid/jname> su -`



> Another question: can I, by using FreeBSD 9 x64, build in the same way FreeBSD 8.2 i386?


Yes, that's possible. Just make sure to use the correct source tree.  




> The reason is that, for example, /usr/include/machine/segments.h on x64 is installed from the x64 platform, meaning no i386 application could be builtd, just because types mismatch (see errors in my first message). That is actually the main reason why I bother about a full valid installation and that's why I need the appropriate source directory from the root.


This is not relevant with regard to the sources in /usr/src/. It's that source tree you can share.


----------



## battleship_potemkin (Mar 16, 2012)

Thanks a lot!
Just a few more questions, if you don't mind.



			
				SirDice said:
			
		

> I'd do `# jexec <jid/jname> su -`



Ok, thank you.




			
				SirDice said:
			
		

> Yes, that's possible. Just make sure to use the correct source tree.



Lovely, what would be the right way to fetch source tree for multiple FreeBSD versions and keep them up to date? I guess it's a CVSup, is that right?

If there is any ready-to-use guide to do that, I would highly appreciate a link or a key-words. If not - that's not a problem - will dig into that.




			
				SirDice said:
			
		

> This is not relevant with regard to the sources in /usr/src/. It's that source tree you can share.



I'm sorry, I feel I didn't put it right.

What I mean is that I can't build i386 binary on x64 platform because most of the headers, that are picked up by a compiler even with non-standard headers specified location, are for the platform I'm building for (x64) - that's why I need a chroot / jail, to ensure compiler will pick up correct headers.


----------



## SirDice (Mar 16, 2012)

battleship_potemkin said:
			
		

> Lovely, what would be the right way to fetch source tree for multiple FreeBSD versions and keep them up to date? I guess it's a CVSup, is that right?


You would need to create multiple trees, unfortunately you can't have different versions in the same tree. But that's not really an issue. Fetching the sources is easily done using csup(1). This works exactly the same as cvsup(1) but doesn't require a port the be installed. For 9.0-RELEASE you would use the RELENG_9_0 tag, for 8.2-RELEASE RELENG_8_2.



> If there is any ready-to-use guide to do that, I would highly appreciate a link or a key-words. If not - that's not a problem - will dig into that.


There's a post detailing getting sources for 9.0-RELEASE. The same principle can be applied to 8.x. Thread 29172



> What I mean is that I can't build i386 binary on x64 platform because most of the headers, that are picked up by a compiler even with non-standard headers specified location, are for the platform I'm building for (x64) - that's why I need a chroot / jail, to ensure compiler will pick up correct headers.



I understood that. But this has nothing to do with the sources in /usr/src/. Since you're going to jexec into a separate environment the files in /usr/include/ will be the correct ones.


----------



## battleship_potemkin (Mar 16, 2012)

SirDice, that's clear now - thanks a lot for your help!


----------

