# sendmail core dumps



## tingo (Apr 3, 2013)

Background: I have a virtual test machine (it runs in VirtualBox, on a FreeBSD host) which currently runs FreeBSD 8.4-PRERELEASE:

```
tingo@kg-v3$ uname -a
FreeBSD kg-v3.kg4.no 8.4-PRERELEASE FreeBSD 8.4-PRERELEASE #0: Wed Apr  3 00:09:32 CEST 2013
     root@kg-v3.kg4.no:/usr/obj/usr/src/sys/GENERIC  i386
```
It used to run FreeBSD 8.2-RELEASE, a standard install, but was upgraded yesterday. For a long time (since June 2011) it was working without problems, running FreeBSD 8.2-RELEASE, with Sendmail working (mostly delivering periodic emails). But some days ago, it started putting these messages in /var/log/messages:

```
Mar 30 03:01:00 kg-v3 kernel: pid 1623 (sendmail), uid 0: exited on signal 11
Mar 30 03:02:11 kg-v3 kernel: pid 1806 (sendmail), uid 0: exited on signal 11
Mar 30 03:02:11 kg-v3 kernel: pid 1859 (sendmail), uid 0: exited on signal 11
```
Yesterday I found PR bin/166248, and decided to upgrade to latest FreeBSD 8-STABLE. Unfortunately, Sendmail still coredumps:

```
Apr  3 22:14:22 kg-v3 kernel: pid 660 (sendmail), uid 0: exited on signal 11
Apr  3 22:14:39 kg-v3 kernel: pid 677 (sendmail), uid 0: exited on signal 11
```
I can make Sendmail dump core in two ways:

```
tingo@kg-v3$ echo test | mail tingo
```
or

```
root@kg-v3# telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Connection closed by foreign host.
```
Has anybody else seen this?

FWIW, I have another virtual machine running on the same host, there Sendmail works:

```
root@kg-v5# uname -a
FreeBSD kg-v5.kg4.no 8.3-STABLE FreeBSD 8.3-STABLE #0: Wed May  2 00:12:29 CEST 2012     root@kg-v5.kg4.no:/usr/obj/usr/src/sys/GENERIC  i386
root@kg-v5# telnet localhost 25                                                                                      Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 kg-v5.kg4.no ESMTP Sendmail 8.14.5/8.14.5; Wed, 3 Apr 2013 22:31:07 +0200 (CEST)
HELO test
250 kg-v5.kg4.no Hello localhost [127.0.0.1], pleased to meet you
QUIT
221 2.0.0 kg-v5.kg4.no closing connection
Connection closed by foreign host.
```


----------



## tingo (Jun 1, 2013)

Update: sendmail on the second machine has started to crash too:

```
root@kg-v5# uname -a
FreeBSD kg-v5.kg4.no 8.3-STABLE FreeBSD 8.3-STABLE #0: Wed May  2 00:12:29 CEST 2012
     root@kg-v5.kg4.no:/usr/obj/usr/src/sys/GENERIC  i386
root@kg-v5# tail /var/log/messages
Jun  1 08:27:57 kg-v5 kernel: pid 3746 (sendmail), uid 0: exited on signal 11
Jun  1 08:57:57 kg-v5 kernel: pid 3797 (sendmail), uid 0: exited on signal 11
Jun  1 09:27:57 kg-v5 kernel: pid 3851 (sendmail), uid 0: exited on signal 11
Jun  1 09:57:57 kg-v5 kernel: pid 3902 (sendmail), uid 0: exited on signal 11
Jun  1 10:27:57 kg-v5 kernel: pid 3955 (sendmail), uid 0: exited on signal 11
Jun  1 10:57:57 kg-v5 kernel: pid 4006 (sendmail), uid 0: exited on signal 11
Jun  1 11:27:57 kg-v5 kernel: pid 4059 (sendmail), uid 0: exited on signal 11
Jun  1 11:57:57 kg-v5 kernel: pid 4136 (sendmail), uid 0: exited on signal 11
Jun  1 12:27:57 kg-v5 kernel: pid 4223 (sendmail), uid 0: exited on signal 11
Jun  1 12:44:17 kg-v5 kernel: pid 4306 (sendmail), uid 0: exited on signal 11
```
Nothing else has changed on the host machine or the guest machine, as far as I can tell.


----------



## wblock@ (Jun 1, 2013)

Memory problems on the host?


----------



## tingo (Jun 2, 2013)

Not likely: the host machine hosts both virtual machines; if memory problems were the reason, both virtual machines should have been affected earlier (they have been started / stopped at different times and in different sequence).


----------



## wblock@ (Jun 2, 2013)

If program and configuration did not change, but the behavior did, the standard culprits are power supply and memory.  If it was load-related, power supply would seem more likely.  But if memory is failing, the VM host may just need to use enough memory to hit the bad spot.  A pass with memtest86 wouldn't hurt.

Otherwise... maybe disk corruption?


----------



## tingo (Jun 3, 2013)

The host runs SETIathome, so I think I would have noticed PSU / load problems there. Maybe memory problems too. I'll try to find time to do a memtest86+ run.

Not really sure how I would go about checking for disk corruption. Wouldn't I also notice disk corruption on the host machine?


----------



## wblock@ (Jun 3, 2013)

tingo said:
			
		

> The host runs setiathome, so I think I would have noticed psu / load problems there. Maybe memory problems too. I'll try to find time to do a memtest86+ run.
> Not really sure how I would go about checking for disk corruption. Wouldn't I also notice disk corruption on the host machine?



Maybe not, depending on where it happens.  Did you accept the default emulated hardware for the VM, particularly the IDE disk controller?  The others are not well-tested on FreeBSD and I had trouble with the SATA controller on earlier releases.


----------



## tingo (Jun 7, 2013)

It's an IDE controller at least (I can't remember if it is the default one, but I only change things if the default doesn't work). From `VBoxManage showvminfo`:


```
Storage Controller Name (0):            IDE Controller
Storage Controller Type (0):            PIIX4
Storage Controller Instance Number (0): 0
Storage Controller Max Port Count (0):  2
Storage Controller Port Count (0):      2
Storage Controller Bootable (0):        on
```


----------



## tingo (Jun 8, 2013)

Ok, it doesn't look like it is disk corruption. I copied the disk image (*.vdi) to another machine and started it with qemu, like this `qemu-system-i386 FreeBSD-v3.vdi` and the virtual machine came up, and showed me that I have 91 unread mails. So it works in qemu, hmm...


----------



## tingo (Jun 9, 2013)

Ok, it is confirmed - Sendmail on both virtual machines works without crashing when I use Qemu instead of VirtualBox. This is tested on the same host, so it looks more like a VirtualBox problem than some problem with the host machine. Unfortunately, I can't get the network interface to pass packets (I'm using tap0 and bridge0 for Qemu), so I can't use Qemu.


----------



## tingo (Jun 10, 2013)

I upgraded VirtualBox on the host to version 4.2.6. Nothing changed - Sendmail still crashes.


----------



## wblock@ (Jun 10, 2013)

If you can duplicate it without giving someone a copy of your VM, the freebsd-emulation would be a good place to start.  It might be an upstream bug, too.


----------



## tingo (Jul 16, 2013)

Yes, I know - but duplicating the issue is hard; the error isn't entirely consistent.

Anyway, I have now upgraded VirtualBox on the host to version 4.2.12

```
root@kg-vm# pv virt*
virtualbox-ose-4.2.12       =  up-to-date with port
virtualbox-ose-kmod-4.2.12  =  up-to-date with port
```
and initial results are good - Sendmail in the virtual machines doesn't crash so far. Here is a test from one of the machines:

```
root@kg-v3# telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 kg-v3.kg4.no ESMTP Sendmail 8.14.7/8.14.7; Tue, 16 Jul 2013 00:54:58 +0200 (CEST)
quit
221 2.0.0 kg-v3.kg4.no closing connection
Connection closed by foreign host.
```
We will see in a few days if the situation changes.


----------



## tingo (Aug 22, 2013)

Well, the machine has now been up for 30 days

```
root@kg-v3# uptime
11:05PM  up 30 days, 19:37, 2 users, load averages: 0.00, 0.00, 0.00
```
and Sendmail still works

```
root@kg-v3# telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 kg-v3.kg4.no ESMTP Sendmail 8.14.7/8.14.7; Thu, 22 Aug 2013 23:04:09 +0200 (CEST)
quit
221 2.0.0 kg-v3.kg4.no closing connection
Connection closed by foreign host.
```


----------

