# ZFS Write Speed Very Inconsistent



## thavinci (Jul 7, 2010)

After many tweaks and issues from Freebsd 8.0 REL all the way up to FreeBSD8.1-RC2 ive been trying to get ZFS actually working like a normal file system. From Commands Freezing up and unable to control pools to kernel panics when drives are added and removed. Oh and kernel panics when a drive becomes physically faulty.

The latest issue i have after getting it running with the occasional kmem_map too small panic is that write speed is VERY eratic, have not tested read speed yet!


```
zpool status                                                                                      Press ? for help
  pool: tank
 state: ONLINE
 scrub: none requested
config:

        NAME                    STATE     READ WRITE CKSUM
        tank                    ONLINE       0     0     0
          raidz1                ONLINE       0     0     0
            label/Seagate1.5_1  ONLINE       0     0     0
            label/Seagate1.5_2  ONLINE       0     0     0
            label/Seagate1.5_3  ONLINE       0     0     0
            label/Seagate1.5_4  ONLINE       0     0     0
            label/WD15EADS      ONLINE       0     0     0
            label/WD15EARS      ONLINE       0     0     0

errors: No known data errors
```


loader.conf looks like:


```
autoboot_delay="1"
aio_load="YES"
ahci_load="YES"
atapicam_load="YES"
vm.kmem_size="3072M"
vfs.zfs.arc_min="1024M"
vfs.zfs.arc_max="2048M"
vfs.zfs.txg.timeout="5"
```

uname -a

```
FreeBSD e-soul.co.za 8.1-RC2 FreeBSD 8.1-RC2 #0: Sun Jul  4 20:08:34 SAST 2010     [email]thavinci@e-
soul.co.za[/email]:/usr/src/sys/amd64/compile/thavinci  amd64
```


Physical Ram is 8Gb.

And here is a graphycal respresentation of the issue using bmon from ports.









As you can see for an average of 5 seconds at a time transfers die down and idle in the kb range then shoot up to 100Mb +-

What on earth could be causing this?!


----------



## Alt (Jul 7, 2010)

http://forums.freebsd.org/showthread.php?t=15647#12


----------



## thavinci (Jul 8, 2010)

i have tried 

```
# sysctl vfs.zfs.txg.write_limit_override=1048576000
```
or even lower this does not effect the results i get, only slows it down a bit.

Ie the bursts only hit 100 or whatever i set it too.

I have confirmed that read speeds with concurrent copies i am able to acheive 100M+ constant performance.


----------



## t1066 (Jul 8, 2010)

You may try to add


```
vfs.zfs.txg.synctime=[I]val[/I]
```

to loader.conf, where _val_ is between 1 and 4.


----------



## Matty (Jul 8, 2010)

thavinci said:
			
		

> i have tried
> 
> ```
> # sysctl vfs.zfs.txg.write_limit_override=1048576000
> ...



have you removed 
	
	



```
vfs.zfs.txg.timeout="5"
```
 from the loader.conf ?

and the WD drives, are these the ones with 4k sectorsize?


----------



## thavinci (Jul 8, 2010)

*Still Slow Write*

I assumed that the two WD15EADS and WD15EARS drives weir the culprits so i did the following setup instead.


zpool status                                                                 Press ? for help
  pool: tank
 state: ONLINE
 scrub: none requested
config:

        NAME                    STATE     READ WRITE CKSUM
        tank                    ONLINE       0     0     0
          raidz1                ONLINE       0     0     0
            label/Seagate1.5_1  ONLINE       0     0     0
            label/Seagate1.5_2  ONLINE       0     0     0
            label/Seagate1.5_3  ONLINE       0     0     0
            label/Seagate1.5_4  ONLINE       0     0     0
            label/WD15EADS.nop  ONLINE       0     0     0
            label/WD15EARS.nop  ONLINE       0     0     0

diskinfo -v label/WD15EADS.nop
label/WD15EADS.nop
        4096            # sectorsize
        1500301909504   # mediasize in bytes (1.4T)
        366284645       # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        363377          # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.
        WD-WCAVY0477467s0       # Disk ident.


diskinfo -v label/WD15EARS.nop
label/WD15EARS.nop
        4096            # sectorsize
        1500301909504   # mediasize in bytes (1.4T)
        366284645       # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        363377          # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.
        WD-WCAVY2852332s0       # Disk ident.








As you can see no difference by addressing the sector issues with these drives.
I am hopeless at this stage and have decided that this must be acceptable performance for zfs on FreeBSD at the momment.


Input?


----------



## thavinci (Jul 8, 2010)

I was posting my response as that one came in! ;>

Yes they are those dreadful 4k Drives.

And how would removing

```
vfs.zfs.txg.timeout="5"
```
Benefit? Everywhere i look they advise specifying that.


*UPDATE:* Seems to change the pausing a little but same thing still applies.
Removing it resulted it taking the default value of 30.

As for 
	
	



```
vfs.zfs.txg.synctime=val
```
Im first trying to find out what that does before testing, what i can say is that it's on 5 at the moment.


----------



## Matty (Jul 8, 2010)

thavinci said:
			
		

> I was posting my response as that one came in! ;>
> 
> Yes they are those dreadful 4k Drives.
> 
> ...



in combination with

```
sysctl vfs.zfs.txg.write_limit_override=1048576000
```
 the lower timeout isn't any useful.

the assumption is that after 5sec a X amount of data is in the txg and should be low enough for the drives to handle. 

what the vfs.zfs.txg.write_limit_override does is that only after Y amount of data the txg gets written. If it takes 25sec to reach that amount doesn't really matter because the Y amount is small enough for the drives to avoid the write stall.


nice thing about the vfs.zfs.txg.write_limit_override is that you can tune it with sysctl so no reboot it needed.
just play with it until you found the right settings. start with 75mb/s and work you way up.

Update: if it's possible. Try to recreate the pool without the WD drive to see if things start to improve.


----------



## thavinci (Jul 8, 2010)

Ok i did remove the 
	
	



```
vfs.zfs.txg.timeout="5"
```

And i did following as more extreme test 
	
	



```
sysctl vfs.zfs.txg.write_limit_override=524288000
```

Issue still exists just not as extreme between the high and low...






Where in the past it would drop into the kb range it now drops into the 1-3 maby 8M range.


----------



## Matty (Jul 8, 2010)

did you try without the WD disks yet?


----------



## thavinci (Jul 8, 2010)

Nope, i can for arguments sake, but other than that. It would defeat the objectives i am trying to meet.

Let me know if you would like me to test to hopefully exclude theme.


*UPDATE:* I will be testing this , just moving data off atm.

I have decided the risk of data loss by using the disks individually with UFS is probably going to outway all the issues i have had and will have with ZFS. I really hope to get it working, but if i don't by end of today, it's back to the tried and tested methods. Will be a pitty to waste 2 weeks of work though! ;p


----------



## Matty (Jul 8, 2010)

thavinci said:
			
		

> Nope, i can for arguments sake, but other than that. It would defeat the objectives i am trying to meet.
> 
> Let me know if you would like me to test to hopefully exclude theme.



you should at least get stable write speed with the speed of 1 disk (90mb/s or so)

If there isn't anything imported on the pool it's quick to test imo but see for you self.


----------



## thavinci (Jul 8, 2010)

Ok now without the 2 suspect WD drives, same issue although less so overall.

This is result without any custom sysctl values at all now as well, ive left everything as defaults


----------



## Matty (Jul 8, 2010)

and with 
	
	



```
sysctl vfs.zfs.txg.write_limit_override
```


----------



## thavinci (Jul 8, 2010)

Ok after continues testing i can't come up with a definite answer to this one.
I tested Seagate on own and problem was there. I applied 
	
	



```
vfs.zfs.txg.write_limit_override=524288000
```
 and it solved it there.

However with multiple simultaneous copies speed went above limit.
I tested 4 seagates a few times and sometimes it was fine other times presented problem.
Ignored this and went on to adding each WD at a time sometimes fine sometimes problem was there.

Seems i could not get constant results.
However one thing i did notice was no matter what i did if i added the WD15EADS drive it was shot and no matter what that result never changed.

So i think i have two problems here. One from the WD15EADS drive and another unknown, as mentioned i have now excepted the risks and will be going back to a traditional filesystem.

I really appreciate the help, but this nasty experience has chased me away from using zfs for a while still.


----------



## t1066 (Jul 8, 2010)

Maybe reduce ncq depth to 10 will help.

Set


```
vfs.zfs.vdev.max_pending=10
```

in loader.conf and reboot.


----------



## thavinci (Jul 13, 2010)

Just an update on this.

I actually didn't give up and made a temporary plan with my data. After waiting almost a full week for new drives to arrive i received two new 1.5Tb Seagate Hard-drives.

I have now replaced both WD in the array and all 6 Hard-Drives are now Seagates. Identical size and model and even firmware version.
So now drives being at fault is out of the window.....


----------



## chrcol (Jul 20, 2010)

here is what I found optimised for 2 drives in a mirror setup 12 gig ram.


```
zfs_load="YES"
vfs.root.mountfrom="zfs:tank/root"
ahci_load="YES"
vm.kmem_size="8192M"
vfs.zfs.vdev.min_pending=4
vfs.zfs.vdev.max_pending=8
vfs.zfs.arc_max="4096M"
vfs.zfs.txg.synctime="1"
```


----------



## thavinci (Jul 23, 2010)

Ok just another update:

Now settings with OS upgraded to 
	
	



```
FreeBSD e-soul.co.za 8.1-RELEASE FreeBSD 8.1-RELEASE #2: Wed Jul 21 22:02:34 SAST 2010     thavinci@e-soul.co.za:/usr/src/sys/amd64/compile/thavinci  amd64
```

and loader.conf

```
autoboot_delay="1"
aio_load="YES"
ahci_load="YES"
atapicam_load="YES"
vm.kmem_size="4096M"
vfs.zfs.arc_min="1024M"
vfs.zfs.arc_max="3072M"
vfs.zfs.txg.timeout="5"
vfs.zfs.txg.synctime="1"
```

And sysctl.conf


```
security.bsd.see_other_uids=0
kern.ipc.maxsockbuf=16777216
kern.ipc.nmbclusters=32768
kern.ipc.somaxconn=32768
kern.maxfiles=65536
kern.maxfilesperproc=32768
kern.maxvnodes=800000
net.inet.tcp.delayed_ack=0
net.inet.tcp.inflight.enable=0
net.inet.tcp.path_mtu_discovery=0
net.inet.tcp.recvbuf_auto=1
net.inet.tcp.recvbuf_inc=524288
net.inet.tcp.recvbuf_max=16777216
net.inet.tcp.recvspace=65536
net.inet.tcp.rfc1323=1
net.inet.tcp.sendbuf_auto=1
net.inet.tcp.sendbuf_inc=524288
net.inet.tcp.sendspace=65536
net.inet.udp.maxdgram=57344
net.inet.udp.recvspace=65536
net.local.stream.recvspace=65536
net.inet.tcp.sendbuf_max=16777216
net.inet.tcp.mssdflt=1460
kern.coredump=0
kern.corefile=/dev/null
#vfs.zfs.txg.write_limit_override=838860800
vfs.zfs.txg.write_limit_override=524288000
```






While writting to volume TwoTB1 which is a zfs volume but no raid just a single device using zfs it seems to illustrate similar behaviour.
I copied a movie onto the volume sustained rates of around 50Mb till near end where it simply halted, throughput dropped in kb range for 15 seconds in this case then continued.


----------



## thavinci (Jul 23, 2010)

Ok now just for even more info on my system:

dmesg:


```
Copyright (c) 1992-2010 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.1-RELEASE #2: Wed Jul 21 22:02:34 SAST 2010
    thavinci@e-soul.co.za:/usr/src/sys/amd64/compile/thavinci amd64
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Core(TM)2 Duo CPU     E7500  @ 2.93GHz (2933.45-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x1067a  Family = 6  Model = 17  Stepping = 10
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x408e3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,XSAVE>
  AMD Features=0x20100800<SYSCALL,NX,LM>
  AMD Features2=0x1<LAHF>
  TSC: P-state invariant
real memory  = 8589934592 (8192 MB)
avail memory = 8193699840 (7814 MB)
ACPI APIC Table: <INTEL  DP35DP  >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
ioapic0: Changing APIC ID to 2
ioapic0 <Version 2.0> irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0: <INTEL DP35DP> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 900
acpi_button0: <Sleep Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pci0: <simple comms> at device 3.0 (no driver attached)
em0: <Intel(R) PRO/1000 Network Connection 7.0.5> port 0x4400-0x441f mem 0xe4300000-0xe431ffff,0xe4320000-0xe4320fff irq 20 at device 25.0 on pci0
em0: Using MSI interrupt
em0: [FILTER]
em0: Ethernet address: 00:1c:c0:2a:c6:e1
uhci0: <Intel 82801I (ICH9) USB controller> port 0x40e0-0x40ff irq 18 at device 26.0 on pci0
uhci0: [ITHREAD]
uhci0: LegSup = 0x2f00
usbus0: <Intel 82801I (ICH9) USB controller> on uhci0
uhci1: <Intel 82801I (ICH9) USB controller> port 0x40c0-0x40df irq 21 at device 26.1 on pci0
uhci1: [ITHREAD]
uhci1: LegSup = 0x2f00
usbus1: <Intel 82801I (ICH9) USB controller> on uhci1
uhci2: <Intel 82801I (ICH9) USB controller> port 0x40a0-0x40bf irq 17 at device 26.2 on pci0
uhci2: [ITHREAD]
uhci2: LegSup = 0x2f00
usbus2: <Intel 82801I (ICH9) USB controller> on uhci2
ehci0: <Intel 82801I (ICH9) USB 2.0 controller> mem 0xe4321c00-0xe4321fff irq 17 at device 26.7 on pci0
ehci0: [ITHREAD]
usbus3: EHCI version 1.0
usbus3: <Intel 82801I (ICH9) USB 2.0 controller> on ehci0
pcib2: <ACPI PCI-PCI bridge> at device 28.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 28.1 on pci0
pci3: <ACPI PCI bus> on pcib3
atapci0: <Marvell 88SX6101 UDMA133 controller> port 0x3018-0x301f,0x3024-0x3027,0x3010-0x3017,0x3020-0x3023,0x3000-0x300f mem 0xe4200000-0xe42001ff irq 17 at device 0.0 on pci3
atapci0: [ITHREAD]
ata2: <ATA channel 0> on atapci0
ata2: [ITHREAD]
pcib4: <ACPI PCI-PCI bridge> at device 28.2 on pci0
pci4: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 28.3 on pci0
pci5: <ACPI PCI bus> on pcib5
pcib6: <ACPI PCI-PCI bridge> at device 28.4 on pci0
pci6: <ACPI PCI bus> on pcib6
atapci1: <JMicron JMB363 UDMA133 controller> port 0x2018-0x201f,0x2024-0x2027,0x2010-0x2017,0x2020-0x2023,0x2000-0x200f mem 0xe4100000-0xe4101fff irq 16 at device 0.0 on pci6
atapci1: [ITHREAD]
ahci0: <JMicron JMB363 AHCI SATA controller> on atapci1
ahci0: [ITHREAD]
ahci0: AHCI v1.00 with 2 3Gbps ports, Port Multiplier supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich0: [ITHREAD]
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich1: [ITHREAD]
ata3: <ATA channel 0> on atapci1
ata3: [ITHREAD]
uhci3: <Intel 82801I (ICH9) USB controller> port 0x4080-0x409f irq 23 at device 29.0 on pci0
uhci3: [ITHREAD]
uhci3: LegSup = 0x2f00
usbus4: <Intel 82801I (ICH9) USB controller> on uhci3
uhci4: <Intel 82801I (ICH9) USB controller> port 0x4060-0x407f irq 19 at device 29.1 on pci0
uhci4: [ITHREAD]
uhci4: LegSup = 0x2f00
usbus5: <Intel 82801I (ICH9) USB controller> on uhci4
uhci5: <Intel 82801I (ICH9) USB controller> port 0x4040-0x405f irq 18 at device 29.2 on pci0
uhci5: [ITHREAD]
uhci5: LegSup = 0x2f00
usbus6: <Intel 82801I (ICH9) USB controller> on uhci5
ehci1: <Intel 82801I (ICH9) USB 2.0 controller> mem 0xe4321800-0xe4321bff irq 23 at device 29.7 on pci0
ehci1: [ITHREAD]
usbus7: EHCI version 1.0
usbus7: <Intel 82801I (ICH9) USB 2.0 controller> on ehci1
pcib7: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci7: <ACPI PCI bus> on pcib7
re0: <RealTek 8169/8169S/8169SB(L)/8110S/8110SB(L) Gigabit Ethernet> port 0x1000-0x10ff mem 0xe4040000-0xe40400ff irq 21 at device 0.0 on pci7
re0: Chip rev. 0x04000000
re0: MAC rev. 0x00000000
miibus0: <MII bus> on re0
rgephy0: <RTL8169S/8110S/8211B media interface> PHY 1 on miibus0
rgephy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
re0: Ethernet address: 00:13:f7:1a:a0:e7
re0: [FILTER]
vgapci0: <VGA-compatible display> port 0x1400-0x147f mem 0xe0000000-0xe3ffffff,0xe4000000-0xe403ffff at device 1.0 on pci7
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci1: <Intel ICH9 AHCI SATA controller> port 0x4428-0x442f,0x4434-0x4437,0x4420-0x4427,0x4430-0x4433,0x4020-0x403f mem 0xe4321000-0xe43217ff irq 21 at device 31.2 on pci0
ahci1: [ITHREAD]
ahci1: AHCI v1.20 with 6 3Gbps ports, Port Multiplier supported
ahcich2: <AHCI channel> at channel 0 on ahci1
ahcich2: [ITHREAD]
ahcich3: <AHCI channel> at channel 1 on ahci1
ahcich3: [ITHREAD]
ahcich4: <AHCI channel> at channel 2 on ahci1
ahcich4: [ITHREAD]
ahcich5: <AHCI channel> at channel 3 on ahci1
ahcich5: [ITHREAD]
ahcich6: <AHCI channel> at channel 4 on ahci1
ahcich6: [ITHREAD]
ahcich7: <AHCI channel> at channel 5 on ahci1
ahcich7: [ITHREAD]
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
atrtc0: <AT realtime clock> port 0x70-0x71,0x74-0x77 irq 8 on acpi0
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f0 irq 6 drq 2 on acpi0
fdc0: [FILTER]
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: [FILTER]
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xca800-0xcd7ff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
ppc0: cannot reserve I/O port range
uart1: <16550 or compatible> at port 0x2f8-0x2ff irq 3 on isa0
uart1: [FILTER]
est0: <Enhanced SpeedStep Frequency Control> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
est1: <Enhanced SpeedStep Frequency Control> on cpu1
p4tcc1: <CPU Frequency Thermal Control> on cpu1
Timecounters tick every 1.000 msec
ipfw2 (+ipv6) initialized, divert enabled, nat enabled, rule-based forwarding enabled, default to deny, logging disabled
load_dn_sched dn_sched FIFO loaded
load_dn_sched dn_sched PRIO loaded
load_dn_sched dn_sched QFQ loaded
load_dn_sched dn_sched RR loaded
load_dn_sched dn_sched WF2Q+ loaded
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 480Mbps High Speed USB v2.0
usbus4: 12Mbps Full Speed USB v1.0
usbus5: 12Mbps Full Speed USB v1.0
usbus6: 12Mbps Full Speed USB v1.0
usbus7: 480Mbps High Speed USB v2.0
ugen0.1: <Intel> at usbus0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen2.1: <Intel> at usbus2
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen3.1: <Intel> at usbus3
uhub3: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
ugen4.1: <Intel> at usbus4
uhub4: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4
ugen5.1: <Intel> at usbus5
uhub5: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus5
ugen6.1: <Intel> at usbus6
uhub6: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus6
ugen7.1: <Intel> at usbus7
uhub7: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus7
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub4: 2 ports with 2 removable, self powered
uhub5: 2 ports with 2 removable, self powered
uhub6: 2 ports with 2 removable, self powered
ad4: 114472MB <Seagate ST3120026A 3.06> at ata2-master UDMA100 
acd0: DVDROM <HL-DT-STDVD-ROM GDR8164B/0L06> at ata2-slave UDMA33 
uhub3: 6 ports with 6 removable, self powered
uhub7: 6 ports with 6 removable, self powered
acd0: FAILURE - INQUIRY ILLEGAL REQUEST asc=0x24 ascq=0x00 
(probe0:ata2:0:1:0): TEST UNIT READY. CDB: 0 0 0 0 0 0 
(probe0:ata2:0:1:0): CAM status: SCSI Status Error
(probe0:ata2:0:1:0): SCSI status: Check Condition
(probe0:ata2:0:1:0): SCSI sense: NOT READY asc:3a,0 (Medium not present)
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <ST32000542AS CC34> ATA-8 SATA 2.x device
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada1 at ahcich1 bus 0 scbus1 target 0 lun 0cd0 at ata2 bus 0 scbus8 target 1 lun 0
cd0: <HL-DT-ST DVD-ROM GDR8164B 0L06> Removable CD-ROM SCSI-0 device 
cd0: 33.000MB/s transfers
cd0: Attempt to query device size failed: NOT READY, Medium not present
```


----------



## thavinci (Jul 23, 2010)

Dmesg Continued:


```
ada1: <ST32000542AS CC34> ATA-8 SATA 2.x device
ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada2 at ahcich2 bus 0 scbus2 target 0 lun 0
ada2: <ST31500341AS CC1H> ATA-8 SATA 2.x device
ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada2: Command Queueing enabled
ada2: 1430799MB (2930277168 512 byte sectors: 16H 63S/T 16383C)
ada3 at ahcich3 bus 0 scbus3 target 0 lun 0
ada3: <ST31500341AS CC1H> ATA-8 SATA 2.x device
ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada3: Command Queueing enabled
ada3: 1430799MB (2930277168 512 byte sectors: 16H 63S/T 16383C)
ada4 at ahcich4 bus 0 scbus4 target 0 lun 0
ada4: <ST31500341AS CC1H> ATA-8 SATA 2.x device
ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada4: Command Queueing enabled
ada4: 1430799MB (2930277168 512 byte sectors: 16H 63S/T 16383C)
ada5 at ahcich5 bus 0 scbus5 target 0 lun 0
ada5: <ST31500341AS CC1H> ATA-8 SATA 2.x device
ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada5: Command Queueing enabled
ada5: 1430799MB (2930277168 512 byte sectors: 16H 63S/T 16383C)
ada6 at ahcich6 bus 0 scbus6 target 0 lun 0
ada6: <ST31500341AS CC1H> ATA-8 SATA 2.x device
ada6: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada6: Command Queueing enabled
ada6: 1430799MB (2930277168 512 byte sectors: 16H 63S/T 16383C)
ada7 at ahcich7 bus 0 scbus7 target 0 lun 0
ada7: <ST31500341AS CC1H> ATA-8 SATA 2.x device
ada7: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada7: Command Queueing enabled
ada7: 1430799MB (2930277168 512 byte sectors: 16H 63S/T 16383C)
SMP: AP CPU #1 Launched!
ugen2.2: <vendor 0x13ba> at usbus2
ukbd0: <vendor 0x13ba Generic USB K/B, class 0/0, rev 1.10/0.01, addr 2> on usbus2
kbd2 at ukbd0
ums0: <vendor 0x13ba Generic USB K/B, class 0/0, rev 1.10/0.01, addr 2> on usbus2
ums0: 3 buttons and [XYZ] coordinates ID=1
Trying to mount root from ufs:/dev/ad4s1a
ZFS filesystem version 3
ZFS storage pool version 14
re0: link state changed to UP
em0: link state changed to UP
acd0: FAILURE - ATA_IDENTIFY status=51<READY,DSC,ERROR> error=4<ABORTED> LBA=0
ahcich1: Timeout on slot 21
ahcich1: is 00000000 cs 00000000 ss ffffffff rs ffffffff tfd 40 serr 00980000
```





And The New Diskinfo:


```
diskinfo -v /dev/ada0
/dev/ada0
        512             # sectorsize
        2000398934016   # mediasize in bytes (1.8T)
        3907029168      # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        3876021         # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.
        6XW1FC77        # Disk ident.
```


```
[root@e-soul /mnt/2tb2/Movies]# diskinfo -v /dev/ada1
/dev/ada1
        512             # sectorsize
        2000398934016   # mediasize in bytes (1.8T)
        3907029168      # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        3876021         # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.
        6XW1QP4R        # Disk ident.
```


```
[root@e-soul /mnt/2tb2/Movies]# diskinfo -v /dev/ada2
/dev/ada2
        512             # sectorsize
        1500301910016   # mediasize in bytes (1.4T)
        2930277168      # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        2907021         # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.
        9VS2CP68        # Disk ident.
```


```
[root@e-soul /mnt/2tb2/Movies]# diskinfo -v /dev/ada3
/dev/ada3
        512             # sectorsize
        1500301910016   # mediasize in bytes (1.4T)
        2930277168      # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        2907021         # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.
        9VS3Z351        # Disk ident.
```


```
[root@e-soul /mnt/2tb2/Movies]# diskinfo -v /dev/ada4
/dev/ada4
        512             # sectorsize
        1500301910016   # mediasize in bytes (1.4T)
        2930277168      # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        2907021         # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.
        9VS262TV        # Disk ident.
```


```
[root@e-soul /mnt/2tb2/Movies]# diskinfo -v /dev/ada5
/dev/ada5
        512             # sectorsize
        1500301910016   # mediasize in bytes (1.4T)
        2930277168      # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        2907021         # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.
        9VS3X19R        # Disk ident.
```


```
[root@e-soul /mnt/2tb2/Movies]# diskinfo -v /dev/ada6
/dev/ada6
        512             # sectorsize
        1500301910016   # mediasize in bytes (1.4T)
        2930277168      # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        2907021         # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.
        9VS3VZLH        # Disk ident.
```


```
[root@e-soul /mnt/2tb2/Movies]# diskinfo -v /dev/ada7
/dev/ada7
        512             # sectorsize
        1500301910016   # mediasize in bytes (1.4T)
        2930277168      # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        2907021         # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.
        9VS3MMTK        # Disk ident.
```

And The Only additional options in my custom kernel:


```
#Firewall
options         IPFIREWALL
options         IPFIREWALL_VERBOSE
options         IPFIREWALL_VERBOSE_LIMIT=500
options         IPFIREWALL_FORWARD
options         IPDIVERT
options         DUMMYNET

options         IPFIREWALL_NAT
options         LIBALIAS

device          ucom            # Generic com ttys
device          umodem          # USB modem support
device          u3g             # USB-based 3G modems (Option, Huawei, Sierra)
```


----------



## thavinci (Jul 23, 2010)

Oh and here is 
zfs get all:


```
NAME          PROPERTY              VALUE                  SOURCE
TwoTB1        type                  filesystem             -
TwoTB1        creation              Fri Jul 16 23:37 2010  -
TwoTB1        used                  624G                   -
TwoTB1        available             1.18T                  -
TwoTB1        referenced            624G                   -
TwoTB1        compressratio         1.00x                  -
TwoTB1        mounted               yes                    -
TwoTB1        quota                 none                   default
TwoTB1        reservation           none                   default
TwoTB1        recordsize            128K                   default
TwoTB1        mountpoint            /mnt/2tb1              local
TwoTB1        sharenfs              off                    default
TwoTB1        checksum              on                     default
TwoTB1        compression           off                    default
TwoTB1        atime                 off                    local
TwoTB1        devices               on                     default
TwoTB1        exec                  on                     default
TwoTB1        setuid                on                     default
TwoTB1        readonly              off                    default
TwoTB1        jailed                off                    default
TwoTB1        snapdir               hidden                 default
TwoTB1        aclmode               groupmask              default
TwoTB1        aclinherit            restricted             default
TwoTB1        canmount              on                     default
TwoTB1        shareiscsi            off                    default
TwoTB1        xattr                 off                    temporary
TwoTB1        copies                1                      default
TwoTB1        version               3                      -
TwoTB1        utf8only              off                    -
TwoTB1        normalization         none                   -
TwoTB1        casesensitivity       sensitive              -
TwoTB1        vscan                 off                    default
TwoTB1        nbmand                off                    default
TwoTB1        sharesmb              off                    default
TwoTB1        refquota              none                   default
TwoTB1        refreservation        none                   default
TwoTB1        primarycache          all                    default
TwoTB1        secondarycache        all                    default
TwoTB1        usedbysnapshots       0                      -
TwoTB1        usedbydataset         624G                   -
TwoTB1        usedbychildren        1.03M                  -
TwoTB1        usedbyrefreservation  0                      -
TwoTB2        type                  filesystem             -
TwoTB2        creation              Fri Jul 16 23:37 2010  -
TwoTB2        used                  500G                   -
TwoTB2        available             1.30T                  -
TwoTB2        referenced            500G                   -
TwoTB2        compressratio         1.00x                  -
TwoTB2        mounted               yes                    -
TwoTB2        quota                 none                   default
TwoTB2        reservation           none                   default
TwoTB2        recordsize            128K                   default
TwoTB2        mountpoint            /mnt/2tb2              local
TwoTB2        sharenfs              off                    default
TwoTB2        checksum              on                     default
TwoTB2        compression           off                    default
TwoTB2        atime                 off                    local
TwoTB2        devices               on                     default
TwoTB2        exec                  on                     default
TwoTB2        setuid                on                     default
TwoTB2        readonly              off                    default
TwoTB2        jailed                off                    default
TwoTB2        snapdir               hidden                 default
TwoTB2        aclmode               groupmask              default
TwoTB2        aclinherit            restricted             default
TwoTB2        canmount              on                     default
TwoTB2        shareiscsi            off                    default
TwoTB2        xattr                 off                    temporary
TwoTB2        copies                1                      default
TwoTB2        version               3                      -
TwoTB2        utf8only              off                    -
TwoTB2        normalization         none                   -
TwoTB2        casesensitivity       sensitive              -
TwoTB2        vscan                 off                    default
TwoTB2        nbmand                off                    default
TwoTB2        sharesmb              off                    default
TwoTB2        refquota              none                   default
TwoTB2        refreservation        none                   default
TwoTB2        primarycache          all                    default
TwoTB2        secondarycache        all                    default
TwoTB2        usedbysnapshots       0                      -
TwoTB2        usedbydataset         500G                   -
TwoTB2        usedbychildren        1.05M                  -
TwoTB2        usedbyrefreservation  0                      -
tank          type                  filesystem             -
tank          creation              Tue Jul 13 18:51 2010  -
tank          used                  4.48T                  -
tank          available             2.16T                  -
tank          referenced            4.27T                  -
tank          compressratio         1.02x                  -
```


----------



## thavinci (Jul 23, 2010)

Continued:


```
tank          mounted               yes                    -
tank          quota                 none                   default
tank          reservation           none                   default
tank          recordsize            128K                   default
tank          mountpoint            /mnt/tank              local
tank          sharenfs              off                    default
tank          checksum              on                     default
tank          compression           off                    default
tank          atime                 off                    local
tank          devices               on                     default
tank          exec                  on                     default
tank          setuid                on                     default
tank          readonly              off                    default
tank          jailed                off                    default
tank          snapdir               hidden                 default
tank          aclmode               groupmask              default
tank          aclinherit            restricted             default
tank          canmount              on                     default
tank          shareiscsi            off                    default
tank          xattr                 off                    temporary
tank          copies                1                      default
tank          version               3                      -
tank          utf8only              off                    -
tank          normalization         none                   -
tank          casesensitivity       sensitive              -
tank          vscan                 off                    default
tank          nbmand                off                    default
tank          sharesmb              off                    default
tank          refquota              none                   default
tank          refreservation        none                   default
tank          primarycache          all                    default
tank          secondarycache        all                    default
tank          usedbysnapshots       0                      -
tank          usedbydataset         4.27T                  -
tank          usedbychildren        215G                   -
tank          usedbyrefreservation  0                      -
tank/Backups  type                  filesystem             -
tank/Backups  creation              Thu Jul 15 18:11 2010  -
tank/Backups  used                  54.5G                  -
tank/Backups  available             2.16T                  -
tank/Backups  referenced            54.5G                  -
tank/Backups  compressratio         1.73x                  -
tank/Backups  mounted               yes                    -
tank/Backups  quota                 none                   default
tank/Backups  reservation           none                   default
tank/Backups  recordsize            128K                   default
tank/Backups  mountpoint            /mnt/tank/Backups      local
tank/Backups  sharenfs              off                    default
tank/Backups  checksum              on                     default
tank/Backups  compression           gzip-9                 local
tank/Backups  atime                 off                    inherited from tank
tank/Backups  devices               on                     default
tank/Backups  exec                  on                     default
tank/Backups  setuid                on                     default
tank/Backups  readonly              off                    default
tank/Backups  jailed                off                    default
tank/Backups  snapdir               hidden                 default
tank/Backups  aclmode               groupmask              default
tank/Backups  aclinherit            restricted             default
tank/Backups  canmount              on                     default
tank/Backups  shareiscsi            off                    default
tank/Backups  xattr                 off                    temporary
tank/Backups  copies                1                      default
tank/Backups  version               3                      -
tank/Backups  utf8only              off                    -
tank/Backups  normalization         none                   -
tank/Backups  casesensitivity       sensitive              -
tank/Backups  vscan                 off                    default
tank/Backups  nbmand                off                    default
tank/Backups  sharesmb              off                    default
tank/Backups  refquota              none                   default
tank/Backups  refreservation        none                   default
tank/Backups  primarycache          all                    default
tank/Backups  secondarycache        all                    default
tank/Backups  usedbysnapshots       0                      -
tank/Backups  usedbydataset         54.5G                  -
tank/Backups  usedbychildren        0                      -
tank/Backups  usedbyrefreservation  0                      -
tank/Docs     type                  filesystem             -
tank/Docs     creation              Thu Jul 15 13:23 2010  -
tank/Docs     used                  302M                   -
tank/Docs     available             2.16T                  -
tank/Docs     referenced            302M                   -
tank/Docs     compressratio         1.26x                  -
tank/Docs     mounted               yes                    -
tank/Docs     quota                 none                   default
tank/Docs     reservation           none                   default
tank/Docs     recordsize            128K                   default
tank/Docs     mountpoint            /mnt/tank/Docs         inherited from tank
tank/Docs     sharenfs              off                    default
tank/Docs     checksum              on                     default
tank/Docs     compression           gzip-9                 local
tank/Docs     atime                 off                    inherited from tank
tank/Docs     devices               on                     default
tank/Docs     exec                  on                     default
tank/Docs     setuid                on                     default
tank/Docs     readonly              off                    default
tank/Docs     jailed                off                    default
tank/Docs     snapdir               hidden                 default
tank/Docs     aclmode               groupmask              default
tank/Docs     aclinherit            restricted             default
tank/Docs     canmount              on                     default
tank/Docs     shareiscsi            off                    default
tank/Docs     xattr                 off                    temporary
tank/Docs     copies                1                      default
tank/Docs     version               3                      -
tank/Docs     utf8only              off                    -
tank/Docs     normalization         none                   -
tank/Docs     casesensitivity       sensitive              -
tank/Docs     vscan                 off                    default
tank/Docs     nbmand                off                    default
tank/Docs     sharesmb              off                    default
tank/Docs     refquota              none                   default
tank/Docs     refreservation        none                   default
tank/Docs     primarycache          all                    default
tank/Docs     secondarycache        all                    default
tank/Docs     usedbysnapshots       0                      -
tank/Docs     usedbydataset         302M                   -
tank/Docs     usedbychildren        0                      -
tank/Docs     usedbyrefreservation  0                      -
```


----------



## thavinci (Jul 23, 2010)

Continued:


```
tank/Nico     type                  filesystem             -
tank/Nico     creation              Wed Jul 14 12:53 2010  -
tank/Nico     used                  136G                   -
tank/Nico     available             2.16T                  -
tank/Nico     referenced            136G                   -
tank/Nico     compressratio         1.65x                  -
tank/Nico     mounted               yes                    -
tank/Nico     quota                 none                   default
tank/Nico     reservation           none                   default
tank/Nico     recordsize            128K                   default
tank/Nico     mountpoint            /mnt/Nico              local
tank/Nico     sharenfs              off                    default
tank/Nico     checksum              on                     default
tank/Nico     compression           gzip-9                 local
tank/Nico     atime                 off                    inherited from tank
tank/Nico     devices               on                     default
tank/Nico     exec                  on                     default
tank/Nico     setuid                on                     default
tank/Nico     readonly              off                    default
tank/Nico     jailed                off                    default
tank/Nico     snapdir               hidden                 default
tank/Nico     aclmode               groupmask              default
tank/Nico     aclinherit            restricted             default
tank/Nico     canmount              on                     default
tank/Nico     shareiscsi            off                    default
tank/Nico     xattr                 off                    temporary
tank/Nico     copies                1                      default
tank/Nico     version               3                      -
tank/Nico     utf8only              off                    -
tank/Nico     normalization         none                   -
tank/Nico     casesensitivity       sensitive              -
tank/Nico     vscan                 off                    default
tank/Nico     nbmand                off                    default
tank/Nico     sharesmb              off                    default
tank/Nico     refquota              none                   default

tank/Nico     refreservation        none                   default
tank/Nico     primarycache          all                    default
tank/Nico     secondarycache        all                    default
tank/Nico     usedbysnapshots       0                      -
tank/Nico     usedbydataset         136G                   -
tank/Nico     usedbychildren        0                      -
tank/Nico     usedbyrefreservation  0                      -
```


----------



## thavinci (Jul 23, 2010)

Continued:


```
tank/Sam      type                  filesystem             -
tank/Sam      creation              Wed Jul 14 13:43 2010  -
tank/Sam      used                  75.3M                  -
tank/Sam      available             2.16T                  -
tank/Sam      referenced            75.3M                  -
tank/Sam      compressratio         1.02x                  -
tank/Sam      mounted               yes                    -
tank/Sam      quota                 none                   default
tank/Sam      reservation           none                   default
tank/Sam      recordsize            128K                   default
tank/Sam      mountpoint            /mnt/Sam               local
tank/Sam      usedbyrefreservation  0                      -
tank/Sam      sharenfs              off                    default
tank/Sam      checksum              on                     default
tank/Sam      compression           gzip-9                 local
tank/Sam      atime                 off                    inherited from tank
tank/Sam      devices               on                     default
tank/Sam      exec                  on                     default
tank/Sam      setuid                on                     default
tank/Sam      readonly              off                    default
tank/Sam      jailed                off                    default
tank/Sam      snapdir               hidden                 default
tank/Sam      aclmode               groupmask              default
tank/Sam      aclinherit            restricted             default
tank/Sam      canmount              on                     default
tank/Sam      shareiscsi            off                    default
tank/Sam      xattr                 off                    temporary
tank/Sam      copies                1                      default
tank/Sam      version               3                      -
tank/Sam      utf8only              off                    -
tank/Sam      normalization         none                   -
tank/Sam      casesensitivity       sensitive              -
tank/Sam      vscan                 off                    default
tank/Sam      nbmand                off                    default
tank/Sam      sharesmb              off                    default
tank/Sam      refquota              none                   default
tank/Sam      refreservation        none                   default
tank/Sam      primarycache          all                    default
tank/Sam      secondarycache        all                    default
tank/Sam      usedbysnapshots       0                      -
tank/Sam      usedbydataset         75.3M                  -
tank/Sam      usedbychildren        0                      -
```


----------



## thavinci (Jul 23, 2010)

And Last:


```
tank/lptdrv   type                  filesystem             -
tank/lptdrv   creation              Wed Jul 14 14:37 2010  -
tank/lptdrv   used                  24.0G                  -
tank/lptdrv   available             2.16T                  -
tank/lptdrv   referenced            24.0G                  -
tank/lptdrv   compressratio         1.04x                  -
tank/lptdrv   mounted               yes                    -
tank/lptdrv   quota                 none                   default
tank/lptdrv   reservation           none                   default
tank/lptdrv   recordsize            128K                   default
tank/lptdrv   mountpoint            /mnt/lptdrv            local
tank/lptdrv   sharenfs              off                    default
tank/lptdrv   checksum              on                     default
tank/lptdrv   compression           gzip-9                 local
tank/lptdrv   atime                 off                    inherited from tank
tank/lptdrv   devices               on                     default
tank/lptdrv   exec                  on                     default
tank/lptdrv   setuid                on                     default
tank/lptdrv   readonly              off                    default
tank/lptdrv   jailed                off                    default
tank/lptdrv   snapdir               hidden                 default
tank/lptdrv   aclmode               groupmask              default
tank/lptdrv   aclinherit            restricted             default
tank/lptdrv   canmount              on                     default
tank/lptdrv   shareiscsi            off                    default
tank/lptdrv   xattr                 off                    temporary
tank/lptdrv   copies                1                      default
tank/lptdrv   version               3                      -
tank/lptdrv   utf8only              off                    -
tank/lptdrv   normalization         none                   -
tank/lptdrv   casesensitivity       sensitive              -
tank/lptdrv   vscan                 off                    default
tank/lptdrv   nbmand                off                    default
tank/lptdrv   sharesmb              off                    default
tank/lptdrv   refquota              none                   default
tank/lptdrv   refreservation        none                   default
tank/lptdrv   primarycache          all                    default
tank/lptdrv   secondarycache        all                    default
tank/lptdrv   usedbysnapshots       0                      -
tank/lptdrv   usedbydataset         24.0G                  -
tank/lptdrv   usedbychildren        0                      -
tank/lptdrv   usedbyrefreservation  0                      -
```


----------



## DutchDaemon (Jul 23, 2010)

Next time, dump it on http://pastie.org


----------



## thavinci (Jul 24, 2010)

Thank You there for the pastie advice 

Another update this morning i woke up to find server crashed once again.


```
Fatal trap 12: page fault while in kernel mode
```


```
current process = 42 (arc_reclaim_thread)
```


----------



## thavinci (Jul 26, 2010)

Also i do get the same issue's with 

```
vfs.zfs.txg.timeout="5"
```
 removed.

In the interim i learnt somewhere that if that is specified zfs will not abide by the 
	
	



```
vfs.zfs.txg.write_limit_override=524288000
```
 setting.

So It's either or. But none have solved my issues.


----------



## danbi (Jul 30, 2010)

I have had rather unstable system, until I followed the advice to declare 1.5 times the RAM as kmem_size. So, for a 8GB RAM I have in /boot/loader.conf


```
vm.kmem_size="12G"
vfs.zfs.arc_max="2G"
```

The arc_max limit is there because the computer is rather memory loaded (servers several diskless KDE desktops). Since this 'fix' no issues with ZFS at all. If your usage is primarily storage, this may not be useful for you. On another (single-user, but rather loaded) system with the same spec I don't limit arc_max without ill effects.

If you need 'stable'  write performance, I guess you should decrease vfs.zfs.txg.write_limit_override until you get to the point where there are no hiccups. You may monitor drive load with gstat and probably tune until load is just under 100% 

Also, the advice to set


```
vfs.zfs.vdev.min_pending=4
vfs.zfs.vdev.max_pending=8
```

is very reasonable. You could even try lower values


----------



## chrcol (Aug 5, 2010)

the min 4 max 8 I found optimal even with AHCI.


----------

