# Buildworld fails -- always on same spot



## GeorgeLinn (Sep 29, 2010)

I am trying to upgrade 8.0 release to 8.1 stable and my buildworld fails in the same exact spot each time.  I initially used cvsup to update the src tree from 8.0 to 8.1.  After a few failed buildworlds I removed the src totally and ran cvsup with an empty /usr/src, buildworld failed after that too.

The system actually reboots when the buildworld fails.  The last few lines of the buildworld putty session shows:


```
cc -O2 -pipe -I. -DIN_GCC -DHAVE_CONFIG_H -DPREFIX=\"/usr/obj/usr/src/tmp/usr\" -
I/usr/obj/usr/src/tmp/usr/src/gnu/usr.bin/cc/cc_tools/../cc_tools -I/usr/src/gnu/usr.bin/cc/cc_tools/../cc_tools -
I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc -I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc/config -
I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/include -
I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/libcpp/include -
I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/libdecnumber -g -DGENERATOR_FILE -DHAVE_CONFIG_H   -
I/usr/obj/usr/src/tmp/legacy/usr/include -c /usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc/genautomata.c
cc -O2 -pipe -I. -DIN_GCC -DHAVE_CONFIG_H -DPREFIX=\"/usr/obj/usr/src/tmp/usr\" -
I/usr/obj/usr/src/tmp/usr/src/gnu/usr.bin/cc/cc_tools/../cc_tools -I/usr/src/gnu/usr.bin/cc/cc_tools/../cc_tools -
I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc -I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc/config -
I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/include -
I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/libcpp/include -
I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/libdecnumber -g -DGENERATOR_FILE -DHAVE_CONFIG_H   -
I/usr/obj/usr/src/tmp/legacy/usr/include  -static -L/usr/obj/usr/src/tmp/legacy/usr/lib -o genautomata genautomata.o rtl.o read-rtl.o ggc-none.o 
vec.o min-insn-modes.o gensupport.o print-rtl.o errors.o libiberty.a -lm
./genautomata /usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc/config/i386/i386.md insn-conditions.md > insn-automata.c
```


----------



## DutchDaemon (Sep 29, 2010)

Insetad of emptying and re-csup'ing /usr/src you better clean out /usr/obj between tries. That's were failed builds end up and are being re-used:

http://forums.freebsd.org/showthread.php?p=53974&#post53974


----------



## GeorgeLinn (Sep 29, 2010)

I had cleant out that folder too.  Forgot to include that in the original post.  Sorry...


----------



## GeorgeLinn (Sep 29, 2010)

I removed everything under /usr/obj and I'm running a `make cleanworld && make cleandir` and will continue with [cmd=]make buildworld[/cmd] I'll keep my fingers crossed.


----------



## fronclynne (Sep 29, 2010)

Pump the output through tee(1) (something like `# make buildworld | tee /var/tmp/buildworldoutfile`) so hopefully you'll get something slightly more meaningful when(if) it barfs.  Also, make sure you have any and all *-j* turned off, else none of your output will be human readable.

I'm betting on hardware overheating, but I'm only staking 2.113 millibeers on it, so I'm not that confident.


----------



## GeorgeLinn (Sep 29, 2010)

After removing everything under /usr/obj and running [cmd=]make cleanworld && make cleandir[/cmd] the process failed in the exact spot as my original post.

I ran the exact process again, this time dumping the output of the buildworld process to a text file, the process failed in the exact spot again according to what was displayed in my putty session, but the end if the text file showed the following: 


```
cc -O2 -pipe -I. -DIN_GCC -DHAVE_CONFIG_H -DPREFIX=\"/usr/obj/usr/src/tmp/usr\" -
I/usr/obj/usr/src/tmp/usr/src/gnu/usr.bin/cc/cc_tools/../cc_tools -I/usr/src/gnu/usr.bin/cc/cc_tools/../cc_tools -
I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc -I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc/config -
I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/include -
I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/libcpp/include -
I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/libdecnumber -g -DGENERATOR_FILE -DHAVE_CONFIG_H   -
I/usr/obj/usr/src/tmp/legacy/usr/include  -static -L/usr/obj/usr/src/tmp/legacy/usr/lib -o genconstants genconstants.o rtl.o read-rtl.o ggc-
none.o vec.o min-insn-modes.o gensupport.o print-rtl.o errors.o libiberty.a -lm
```


----------



## wblock@ (Sep 29, 2010)

fronclynne said:
			
		

> Pump the output through tee(1) (something like `# make buildworld | tee /var/tmp/buildworldoutfile`) so hopefully you'll get something slightly more meaningful when(if) it barfs.  Also, make sure you have any and all *-j* turned off, else none of your output will be human readable.



script(1) catches everything (including stderr): http://forums.freebsd.org/showthread.php?t=17309

#include custom-cflags-warning


----------



## GeorgeLinn (Sep 29, 2010)

When using script, here is the that last few lines captured before the process stops and the system reboots:

```
cc -O2 -pipe -I. -DIN_GCC -DHAVE_CONFIG_H -DPREFIX=\"/usr/obj/usr/src/tmp/usr\" -
I/usr/obj/usr/src/tmp/usr/src/gnu/usr.bin/cc/cc_tools/../cc_tools -I/usr/src/gnu/usr.bin/cc/cc_tools/../cc_tools -
I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc -I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc/config -
I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/include -
I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/libcpp/include -
I/usr/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/libdecnumber -g -DGENERATOR_FILE -DHAVE_CONFIG_H   -I/u
```


----------



## aragon (Sep 29, 2010)

If your system is rebooting my first suspicion is either insufficient cooling, or bad power supply.  What kind of system do you have?  Many CPUs reboot themselves when they're overheating.


----------



## GeorgeLinn (Sep 29, 2010)

It is a dual processor Compaq DL 320 or 360 pentium III with ~ 1 gig of RAM.  My machine never reboots at any other time besides the buildworld process.  If it were a bad piece of hardware or heating issue, that would not explain why the buildworld process fails at the exact-same-spot each time, I would think the crashes would be a bit more random during the buildworld process.  No?

I guess I will just install 8.1 from scratch and then try a buildworld and see if I run into the same issue...

Thanks.


----------



## jb_fvwm2 (Sep 29, 2010)

*IF* you can underclock the machine in the bios, even
slightly, that *may* fix the buildworld issue.  (Just in
case it occurs also on 8.1)


----------



## Ralph_Ellis (Sep 29, 2010)

*Buildworld fails*

Could you post the contents of your /etc/make.conf ?
I have found that buildworld often succeeds only with an empty /etc/make.conf file. Any customizations can trigger errors. 
Strangely enough, you can often set custom options while doing a buildkernel without problems. 
Good luck.


----------



## DutchDaemon (Sep 30, 2010)

GeorgeLinn, start using proper formatting. Unformatted output and commands look bad.


----------



## GeorgeLinn (Sep 30, 2010)

My /etc/make.conf file just has


```
PERL_VERSION=5.8.9
```

I tried `# make â€“j1 buildworld`thinking this would throttle the build process, but that did not help.

I also disabled one of the CPUâ€™s by adding 
	
	



```
hint.lapic.0.disabled=1
```
 to /boot/loader.conf and while that did disable one of the the CPU's it did not help the build process.

Lastly I tried limiting the amount of RAM that FreeBSD uses to 384M by adding 
	
	



```
hw.physmem="384m"
```
 to /boot/defaults/loader.conf and that failed to help the make buildworld process too.

I will install 8.1 tomorrow and see if the `# make buildworld`process works.


Thanks!


----------



## GeorgeLinn (Oct 1, 2010)

I installed 8.1 from CD and that went fine.  Before getting around to a `buildworld`
I tried installing some ports that were installed in 8.0, the smaller ports installed fine, but the first port that compiled for more that 10 minutes crashed the machine.  It must be failing equipment.  Thanks for your time everyone.  Much appreciated.

George


----------



## jb_fvwm2 (Oct 1, 2010)

BTW another alternative to underclocking is installing a
pci-slot exhaust fan.  (Worked here for a long while, 
seemed to halve the spontaneous reboots).  (If your computer
case has not enough airflow near the CPU).


----------



## wblock@ (Oct 1, 2010)

It really doesn't sound like a heat problem, which would likely vary in when the reboot happens.  Bad RAM seems more likely, although still not something that would have a problem at the same spot every time.  Maybe a processor bug?  http://news.cnet.com/Potential-Pentium-III-bug-easy-to-diagnose/2100-1040_3-240384.html


----------



## Galactic_Dominator (Oct 2, 2010)

I've seen similar behavior before.  Try booting to single user mode and run `# fsck -y`.  Probably wouldn't hurt to run it twice, sometimes one is not enough.


----------



## GeorgeLinn (Oct 2, 2010)

Ok.  I took the machine apart and cleaned out a bunch of dust bunnies and removed (2) 128MB sticks of RAM, leaving in a single 512MB stick.  I can now compile all of my usual ports and buildworld completes too!  I initially did not think this was RAM related because of the way buildworld was failing in the exact place over-and-over, I was convinced it was something being caused by the build process itself.

After thinking about this problem knowing the cause was bad RAM it makes sense to me now.  After the OS loads, and the services load, there is not much else at all being executed or loaded on the machine, very static situation.  This is why the buildworld process failed in the same spot over-and-over.   During the troubleshooting process, if I had executed\loaded anything additional\different prior to the each subsequent buildworld test, the machine would have consumed additional RAM prior to the build process and failed sooner each time because the bad area in the RAM module would have been reached quicker, causing the machine to reset at earlier locations while building world.  

Long live my Pentium III.  Thanks again everyone.


----------

