# Problems at boot-time with USB hard drive



## impulse (May 9, 2015)

I'm running FreeBSD 10.1 on a Intel NUC (Haswell i3) basically serving as a headless server.

For some additional storage I have an USB 3.0 hard drive connected to the system. The hard drive is encrypted with GELI. 

My problem is that the hard drive doesn't get identified by FreeBSD if it's connected at boot-time. However, manually disconnecting and reconnecting the device (after booting) makes it identified and usable by the system. After reconnecting the device it works perfect.

This is really annoying as I don't have direct physical access to the server.

Boot output (notice the USB_ERR_TIMEOUT):


```
Copyright (c) 1992-2014 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.1-RELEASE #1: Fri May  8 16:59:09 CEST 2015
    root@nuc:/usr/obj/usr/src/sys/XXX amd64
FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512
CPU: Intel(R) Core(TM) i3-4010U CPU @ 1.70GHz (1696.11-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x40651  Family = 0x6  Model = 0x45  Stepping = 1
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x7ffafbbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,<b11>,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x21<LAHF,ABM>
  Structured Extended Features=0x27ab<FSGSBASE,TSCADJ,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
  TSC: P-state invariant, performance statistics
real memory  = 4294967296 (4096 MB)
avail memory = 4042326016 (3855 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <INTEL  D34010WY>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 SMT threads
cpu0 (BSP): APIC ID:  0
cpu1 (AP): APIC ID:  1
cpu2 (AP): APIC ID:  2
cpu3 (AP): APIC ID:  3
ACPI Error: Gpe0Block - 32-bit FADT register is too long (32 bytes, 256 bits) to convert to GAS struct - 255 bits max, truncating (20130823/tbfadt-223)
ioapic0 <Version 2.0> irqs 0-39 on motherboard
random: <Software, Yarrow> initialized
kbd1 at kbdmux0
cryptosoft0: <software crypto> on motherboard
aesni0: <AES-CBC,AES-XTS> on motherboard
acpi0: <INTEL D34010WY> on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 550
Event timer "HPET1" frequency 14318180 Hz quality 440
Event timer "HPET2" frequency 14318180 Hz quality 440
Event timer "HPET3" frequency 14318180 Hz quality 440
Event timer "HPET4" frequency 14318180 Hz quality 440
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atrtc0: Warning: Couldn't map I/O.
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
vgapci0: <VGA-compatible display> port 0xf000-0xf03f mem 0xf7800000-0xf7bfffff,0xe0000000-0xefffffff irq 16 at device 2.0 on pci0
vgapci0: Boot video device
hdac0: <Intel Haswell HDA Controller> mem 0xf7d34000-0xf7d37fff irq 16 at device 3.0 on pci0
xhci0: <Intel Panther Point USB 3.0 controller> mem 0xf7d20000-0xf7d2ffff irq 16 at device 20.0 on pci0
xhci0: 32 byte context size.
xhci0: Port routing mask set to 0xffffffff
usbus0 on xhci0
pci0: <simple comms> at device 22.0 (no driver attached)
em0: <Intel(R) PRO/1000 Network Connection 7.4.2> port 0xf080-0xf09f mem 0xf7d00000-0xf7d1ffff,0xf7d3c000-0xf7d3cfff irq 20 at device 25.0 on pci0
em0: Using an MSI interrupt
em0: Ethernet address: XX:XX:XX:XX:XX:XX
hdac1: <Intel Lynx Point-LP HDA Controller> mem 0xf7d30000-0xf7d33fff irq 22 at device 27.0 on pci0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> irq 19 at device 28.3 on pci0
pci2: <ACPI PCI bus> on pcib2
ath0: <Atheros 9280> mem 0xf7c00000-0xf7c0ffff irq 19 at device 0.0 on pci2
ath0: [HT] enabling HT modes
ath0: [HT] 1 stream STBC receive enabled
ath0: [HT] 1 stream STBC transmit enabled
ath0: [HT] 2 RX streams; 2 TX streams
ath0: AR9280 mac 128.2 RF5133 phy 13.0
ath0: 2GHz radio: 0x0000; 5GHz radio: 0x00c0
ehci0: <EHCI (generic) USB 2.0 controller> mem 0xf7d3b000-0xf7d3b3ff irq 23 at device 29.0 on pci0
usbus1: EHCI version 1.0
usbus1 on ehci0
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci0: <Intel Lynx Point-LP AHCI SATA controller> port 0xf0d0-0xf0d7,0xf0c0-0xf0c3,0xf0b0-0xf0b7,0xf0a0-0xf0a3,0xf060-0xf07f mem 0xf7d3a000-0xf7d3a7ff irq 19 at device 31.2 on pci0
ahci0: AHCI v1.30 with 2 6Gbps ports, Port Multiplier not supported
ahcich3: <AHCI channel> at channel 3 on ahci0
acpi_button0: <Power Button> on acpi0
acpi_tz0: <Thermal Zone> on acpi0
acpi_tz1: <Thermal Zone> on acpi0
orm0: <ISA Option ROM> at iomem 0xcf000-0xcffff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
fdc0: <Enhanced floppy controller> at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0
ppc0: cannot reserve I/O port range
uart0: <16550 or compatible> at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
est1: <Enhanced SpeedStep Frequency Control> on cpu1
p4tcc1: <CPU Frequency Thermal Control> on cpu1
est2: <Enhanced SpeedStep Frequency Control> on cpu2
p4tcc2: <CPU Frequency Thermal Control> on cpu2
est3: <Enhanced SpeedStep Frequency Control> on cpu3
p4tcc3: <CPU Frequency Thermal Control> on cpu3
Timecounters tick every 1.000 msec
IPsec: Initialized Security Association Processing.
hdacc0: <Intel Haswell HDA CODEC> at cad 0 on hdac0
hdaa0: <Intel Haswell Audio Function Group> at nid 1 on hdacc0
pcm0: <Intel Haswell (HDMI/DP 8ch)> at nid 3 on hdaa0
hdacc1: <Realtek (0x0283) HDA CODEC> at cad 0 on hdac1
hdaa1: <Realtek (0x0283) Audio Function Group> at nid 1 on hdacc1
pcm1: <Realtek (0x0283) (Left Analog)> at nid 33 and 25 on hdaa1
random: unblocking device.
usbus0: 5.0Gbps Super Speed USB v3.0
usbus1: 480Mbps High Speed USB v2.0
ugen0.1: <0x8086> at usbus0
uhub0: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
uhub0: 13 ports with 13 removable, self powered
ada0 at ahcich3 bus 0 scbus0 target 0 lun 0
ada0: <INTEL SSDMCEAC060B3 LLLi> ATA-9 SATA 3.x device
ada0: Serial Number CVLI327500G7060K
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 57241MB (117231408 512 byte sectors: 16H 63S/T 16383C)
ada0: Previously was known as ad4
SMP: AP CPU #1 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #2 Launched!
Timecounter "TSC" frequency 1696110024 Hz quality 1000
uhub1: 2 ports with 2 removable, self powered
ugen0.2: <Cypress> at usbus0
ukbd0: <EndPoint1 Interrupt> on usbus0
kbd0 at ukbd0
ugen1.2: <vendor 0x8087> at usbus1
uhub2: <vendor 0x8087 product 0x8000, class 9/0, rev 2.00/0.04, addr 2> on usbus1
uhub2: 8 ports with 8 removable, self powered
usb_alloc_device: set address 3 failed (USB_ERR_TIMEOUT, ignored)
Enter passphrase for ada0p4: usbd_setup_device_desc: getting device descriptor at addr 3 failed, USB_ERR_TIMEOUT
usbd_req_re_enumerate: addr=3, set address failed! (USB_ERR_TIMEOUT, ignored)
usbd_setup_device_desc: getting device descriptor at addr 3 failed, USB_ERR_TIMEOUT
usbd_req_re_enumerate: addr=3, set address failed! (USB_ERR_TIMEOUT, ignored)
usbd_setup_device_desc: getting device descriptor at addr 3 failed, USB_ERR_TIMEOUT
usbd_req_re_enumerate: addr=3, set address failed! (USB_ERR_TIMEOUT, ignored)
usbd_setup_device_desc: getting device descriptor at addr 3 failed, USB_ERR_TIMEOUT
usbd_req_re_enumerate: addr=3, set address failed! (USB_ERR_TIMEOUT, ignored)
usbd_setup_device_desc: getting device descriptor at addr 3 failed, USB_ERR_TIMEOUT
ugen0.3: <Unknown> at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
ugen0.3: <Ralink> at usbus0
GEOM_ELI: Wrong key for ada0p4. Tries left: 2.
Enter passphrase for ada0p4: GEOM_ELI: Device ada0p4.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
Trying to mount root from ufs:/dev/ada0p4.eli []...
GEOM_ELI: Device ada0p3.eli created.
GEOM_ELI: Encryption: AES-XTS 128
GEOM_ELI:     Crypto: hardware
bridge0: Ethernet address: XX:XX:XX:XX:XX:XX
wlan0: Ethernet address: XX:XX:XX:XX:XX:XX
ums0: <EP2 Interrupt> on usbus0
ums0: 3 buttons and [XYZ] coordinates ID=1
run0: <1.0> on usbus0
bridge0: error setting interface capabilities on vlan20
run0: MAC/BBP RT3572 (rev 0x0223), RF RT3052 (MIMO 2T2R), address 10:bf:48:5a:e7:4d
wlan1: Ethernet address: XX:XX:XX:XX:XX:XX
```

After disconnecting the harddrive and reconnecting it:


```
ugen0.4: <GODO> at usbus0
umass0: <GODO USB3.0 External HDD, class 0/0, rev 3.00/1.00, addr 4> on usbus0
umass0:  SCSI over Bulk-Only; quirks = 0x0100
umass0:1:0:-1: Attached to scbus1
da0 at umass-sim0 bus 0 scbus1 target 0 lun 0
da0: <Hitachi HTS545050A7E380 0> Fixed Direct Access SCSI-6 device
da0: Serial Number 20140100003B
da0: 400.000MB/s transfers
da0: 476940MB (976773168 512 byte sectors: 255H 63S/T 60801C)
da0: quirks=0xa<NO_6_BYTE,4K>
GEOM_ELI: Device da0.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
```

Anyone has any insight on how to solve it? An acceptable solution for me would be if I could do the "reconnecting part" via command line/SSH. The device doesn't have to be identified at boot.


----------



## Crivens (May 9, 2015)

Maybe you could do that using `camcontrol rescan all` from /etc/rc.local. If that file does not exist, you can create it.


----------



## impulse (May 13, 2015)

Crivens,

Thanks for your support.

I ran the suggested command:


```
camcontrol rescan all
Re-scan of bus 0 was successful
```

No new da0-drive though, and no messages in `dmesg`/syslog.

Any other suggestions?


----------



## protocelt (May 13, 2015)

Since USB drives use the SCSI subsystem you could try increasing the delay to allow the kernel more time for discovery. 
	
	



```
kern.cam.scsi_delay="3000"
```
 I think the default is 2 seconds(2000 ms). It may or may not work. External USB drives under FreeBSD aren't always very reliable. Using eSATA is probably a better choice if viable.


----------



## Crivens (May 15, 2015)

Maybe you need to extract the USB stuff into modules so you can unload and re-load them when the system is up. That should get the drives recognized.


----------



## wblock@ (May 15, 2015)

It's possible that resetting the device with usbconfig(8) would work.


----------

