# Problem installing a 4 TB Hitachi in Freebsd 11.2



## mrredeyeflight (Mar 1, 2019)

Hello

I am having  problem Installing a 4 TB Hitachi internal Harddrive into    a Dell Precision workstation running Freebsd 11.2 P9. I have followed the instructions in the Handbook. I see the drive listed in gpart show as da1. I also see the drive listed with camcontrol devlist -v. When I enter fstyp /dev/da1  I get filesystem not recognized. I want to destroy the drive using gpart destroy,  I get the error messgage, gpart: Device busy. 
When I boot into safe mode and try gpart destroy, I get he same error message gpart: Device busy.
Sometimes I get the message from gpart show that the da1 GPT (3.6T) [CORUPT]  I think the file system is not set up.  Is that because there is no file syswtem on the Harddrive?
I followed the Handbook when I installed the hard drive.
ANy ideas to get this HardDrive working properly in Freebsd 11.2 are welcome.


----------



## Wozzeck.Live (Mar 1, 2019)

Hum.... drives > 2 TB may cause many many issues. Sometimes drive appears but for a very limited capacity, sometimes it seems to work but run into weird problems and you loose data etc etc

1) if you run a 32 bit OS, the OS will see a maximumm of 2 TB ... in the better case. In the worst case you can't access the drive

2) da0 means you have put your drive in an enclosure. This is an USB drive. If you have done that by yourself, using your own enclosure, be aware that there are a lot of USB adaptor (so this is not related to the USB2/3 controller on the motherboard), that doesn't work properly with > 2 TB drives. I can confirm, I have had recently the problem with a 3 TB drive. The faulty USB adaptator has been tested under Windows and FreeBSD, same result... strange errors, I even thought the drive was dead. Using a more recent USB3 adaptator, fixed the issue. The faulty USB adaptator works perfectly with any drive <= 2 TB

3) Even running a 64 bit OS, some old SATA2 controller doesn't recognize > 2TB drives.
On one of my old machine connecting the 3 TB drive to the onboard intel Rapid Storage caused random boot issues, even if the boot drive selected was another drive < 2 TB (booting on > 2 TB drive is only possible with 64 bit OSes and UEFI bios) connected to another ports... but on the same controller.

I have had to plug an additional Sata III controller to connect the 3 TB drive, and I set the boot on the onboard intel.

On another computer, an old PCI SATA 2 Promise controller didn't recognize correctly the drive causing weird error as you describe


----------



## VladiBG (Mar 1, 2019)

did you try `gpart destroy -F da1`


----------



## Emrion (Mar 1, 2019)

> 1) if you run a 32 bit OS, the OS will see a maximumm of 2 TB ... in the better case. In the worst case you can't access the drive


I can't verify but I don't think so. It's more PC hardware and partition scheme dependant.

da0 could be also a SCSI device.

IMO, the first thing to do, if possible, is to verify if the disk is well recognised in term of size in the BIOS/UEFI.


----------



## Bobi B. (Mar 1, 2019)

Usually fstyp(8) should be ran against a partition, not a RAW device; depending on partitioning scheme `fstyp /dev/da1p1` (GPT) or `fstyp /dev/da1s1` (MBR).

gpart(8) `destroy` probably fails as partitions still exists -- i.e. partition table is not empty. Either do as VladiBG said: use `-F` flag to force, or delete partitions first.

One way to use disk hard drive with FreeBSD would be to destroy partition table, create a new GPT one (IIRC MBR doesn't work for drives above 2 TB) (`gpart create -s gpt da1`), add a new partiton (`gpart add -t freebsd-ufs -a 1M -l data /dev/da1`), create a filesystem ("format" it, `newfs -L data -jU /dev/da1p1`) and mount it (`mount /dev/ufs/data /mnt` or `mount /dev/da1p1 /mnt`). When you're happy add an entry to /etc/fstab.

Be careful if you copy/paste above commands for not to destroy something important.


----------



## olli@ (Mar 1, 2019)

Wozzeck.Live said:


> Hum.... drives > 2 TB may cause many many issues.


Actually that should be a thing of the past.
Drives of _any_ size should work fine if you use GPT partitioning (which is the case here). The only thing that might cause problems is if you want to boot from the drive (_not_ the case here) _and_ your BIOS is quite old _and_ your boot partition is not contained within the first 2 TB of the drive.


> 1) if you run a 32 bit OS, the OS will see a maximumm of 2 TB ... in the better case. In the worst case you can't access the drive


I'm sorry, but that's nonsense. Whether the OS is 32 bit or 64 has absolutely nothing to do with the maximum size of disks supported. You can access a 12 TB drive with FreeBSD/i386 without any problems.


> 2) da0 means you have put your drive in an enclosure. This is an USB drive.


No, not necessarily. “da0” means that the disk was attached via the SCSI disk driver of the CAM subsystem. This is the case for SCSI and SAS disks, whether internal or external, and it's also the case for storage devices attached via USB. It is (was) also the case when attaching ATA / ATAPI disks with ata(4) using the deprecated kernel options “device atapicam” and/or “options ATA_CAM” (these don't work anymore in FreeBSD 12, I think).


> booting on > 2 TB drive is only possible with 64 bit OSes and UEFI bios


No, I've got an old non-UEFI server that has two disks (4 TB and 12 TB) and boots just fine. I updated it from i386 to amd64 not too long ago; it didn't make a difference for booting. Just make sure you use a GPT and install a “protective MBR” (PMBR) that will be used by the legacy BIOS for booting. Also, the boot partition should be placed at the beginning of the disk. See the gptboot(8) manual page for details.


----------



## olli@ (Mar 1, 2019)

mrredeyeflight said:


> I am having  problem Installing a 4 TB Hitachi internal Harddrive into a Dell Precision workstation running Freebsd 11.2 P9. I have followed the instructions in the Handbook. I see the drive listed in gpart show as da1.


Is that a SCSI disk? I'm asking because of the device name. For a normal SATA disk it should be `ada`, not `da`. Another explanation would be that the internal connector is really a USB adapter, but that would be rather unusual.


> I also see the drive listed with camcontrol devlist -v. When I enter fstyp /dev/da1 I get filesystem not recognized.


That's expected because there is no file system on `da1`, because that's the device representing the whole disk. The fstyp(8) command can only be used for partitions that contain a file system.
What is the output from `gpart show`, exactly?
If it shows any partitions, you can try to use the `fstyp` command with those to find out if they contain a file system (i. e. if they have already been formatted). Or you can just try to mount them – if that works, they obviously contain a file system.


> I want to destroy the drive using gpart destroy,  I get the error messgage, gpart: Device busy.


That's normal because there are still partitions. You cannot destroy a partition table with existing entries – you have to delete those partitions first.
However, if there are already partitions (one or more), and you're satisfied with the layout (e. g. the sizes of the partitions), then there is no reason to delete or destroy anything. If the partitions contain file systems, they are ready to be mounted. Otherwise you need to create file systems on them first (sometimes known as “formatting”). To create UFS file systems, you can use the newfs(8) command.


----------



## mrredeyeflight (Mar 1, 2019)

VladiBG said:


> did you try `gpart destroy -F da1`


It stil shows device busy.


----------



## mrredeyeflight (Mar 1, 2019)

olli@ said:


> Is that a SCSI disk? I'm asking because of the device name. For a normal SATA disk it should be `ada`, not `da`. Another explanation would be that the internal connector is really a USB adapter, but that would be rather unusual.
> 
> That's expected because there is no file system on `da1`, because that's the device representing the whole disk. The fstyp(8) command can only be used for partitions that contain a file system.
> What is the output from `gpart show`, exactly?
> ...


This is what `gpart show`

```
-BSD ~]# gpart show
=>       63  781422705  da0  MBR  (373G)
         63          1       - free -  (512B)
         64  781422703    1  freebsd  [active]  (373G)
  781422767          1       - free -  (512B)

=>        0  781422703  da0s1  BSD  (373G)
          0  773849088      1  freebsd-ufs  (369G)
  773849088    7573614      2  freebsd-swap  (3.6G)
  781422702          1         - free -  (512B)

=>        40  7814037088  da1  GPT  (3.6T) [CORRUPT]
          40        2008       - free -  (1.0M)
        2048  7814033408    1  freebsd-ufs  (3.6T)
  7814035456        1672       - free -  (836K)
```


----------



## mrredeyeflight (Mar 1, 2019)

Is there a list of what Dell Precision T3600 work with internal harddrives and wheicj does not? I don't think that is in the Handbook?


----------



## mrredeyeflight (Mar 1, 2019)

olli@ said:


> Is that a SCSI disk? I'm asking because of the device name. For a normal SATA disk it should be `ada`, not `da`. Another explanation would be that the internal connector is really a USB adapter, but that would be rather unusual.
> 
> That's expected because there is no file system on `da1`, because that's the device representing the whole disk. The fstyp(8) command can only be used for partitions that contain a file system.
> What is the output from `gpart show`, exactly?
> ...


The Harddrive is not s SCSI drive it is a SATA drive. It could be that the HardDrive controller does not work with the 4 TB HD. I have upgraded the system BIOS to the current BIOS from Dell.

Thank you for your help.


----------



## VladiBG (Mar 1, 2019)

try with dd and wipe the first and last 512 sectors from the drive then reboot.


----------



## olli@ (Mar 1, 2019)

mrredeyeflight said:


> This is what `gpart show`
> 
> ```
> -BSD ~]# gpart show
> ...


Okay. That looks good, actually.
The “corrupt” means that the backup copy of the GPT metadata is incorrect. It can be fixed with `gpart recover da1`.
Finally, try the command `fstyp da1p1`. What is the output? If it is UFS / FFS, you should be able to mount the partition. Just try it.
Otherwise you need to create a file system on /dev/da1p1, for example with newfs(8) to create a UFS file system.


----------



## mrredeyeflight (Mar 2, 2019)

When I enter the commad fstyp da1p1
I get  fstyp: da1p1: No such file or directory


----------



## mrredeyeflight (Mar 2, 2019)

Hello
What i enter /etc/fstab  I get 
Device    Mountpoint    FStype    Options    Dump    Pass#
/dev/da0s1a    /        ufs    rw    1    1
/dev/da0s1b    none        swap    sw    0    0

proc        /proc        procfs    rw    0    0
linprocfs   /compat/linux/proc    linprocfs    rw    0    0
tmpfs    /compat/linux/dev/shm    tmpfs    rw,mode=1777    0    0
 I dont see an enry for the 4 TB internal HD da1  ?
SHould I enter /dev/da1    /newdisk    ufs    rw    2    2   into /etc/fstab?

Thank you for any help?


----------



## olli@ (Mar 2, 2019)

mrredeyeflight said:


> When I enter the commad fstyp da1p1
> I get  fstyp: da1p1: No such file or directory


Did you run the recover command first that I mentioned? Was it successful? Did the “corrupted” tag disappear from the output of `gpart show`?
What does `ls -l /dev/da1*` display?


----------



## mrredeyeflight (Mar 3, 2019)

gpart show
=>       63  781422705  da0  MBR  (373G)
         63          1       - free -  (512B)
         64  781422703    1  freebsd  [active]  (373G)
  781422767          1       - free -  (512B)

=>        0  781422703  da0s1  BSD  (373G)
          0  773849088      1  freebsd-ufs  (369G)
  773849088    7573614      2  freebsd-swap  (3.6G)
  781422702          1         - free -  (512B)

=>        40  7814037088  da1  GPT  (3.6T)
          40        2008       - free -  (1.0M)
        2048  7814033408    1  freebsd-ufs  (3.6T)
  7814035456        1672       - free -  (836K)

ls -l /dev/da1*
crw-r-----  1 root  operator  0x6a Mar  3 12:30 /dev/da1
crw-r-----  1 root  operator  0x6e Mar  3 12:30 /dev/da1p1


----------



## olli@ (Mar 3, 2019)

Ok, very good. Now try: `fstyp /dev/da1p1`
(Note that you have to specify the full path, including the /dev directory.)


----------



## mrredeyeflight (Mar 4, 2019)

fstyp /dev/da1p1
ufs

Also every time I book the freebsd system. I have to use gpart recover da1  ?

I really appreicate your and others help


----------



## olli@ (Mar 4, 2019)

mrredeyeflight said:


> fstyp /dev/da1p1
> ufs


Ok, then the partition already contains a file system. You should be able to mount it, for example on /mnt:
`mount /dev/da1p1 /mnt`
To mount it automatically after reboot, add an entry to the /etc/fstab file, see the fstab(5) manual page.
Note that the mount point has to exist, i.e. you need to create the directory beforehand where you want to mount the file system.



> Also every time I book the freebsd system. I have to use gpart recover da1  ?


No, you don't have to. Once you recovered it, it should be fixed.
Or are you saying that it is corrupted again each time you reboot? That shouldn't happen, really. That would be very strange.


----------



## mrredeyeflight (Mar 5, 2019)

Hello
It is corrupt after a reboot. It fixes just fine, It just doesn't stay fixed.
There is nothing in the etc/fstab for the da1 internal hard drive.
A while back I enered   /dev/da1p1    /newdisk    ufs    rw    2    2
After I entered it, I was unable to get the machine to boot Freebsd. I had to boot into safe mode. Is  This  the correct entry for etc/fstab correct?      /dev/da1p1    /newdisk    ufs    rw    2    2


----------



## olli@ (Mar 5, 2019)

mrredeyeflight said:


> It is corrupt after a reboot. It fixes just fine, It just doesn't stay fixed.


That's very strange. I'm sorry I don't have an explanation for that behavior.
Maybe you should open a new thread for that problem, so it gets more exposure. Maybe someone else can help you.



> There is nothing in the etc/fstab for the da1 internal hard drive.
> A while back I enered   /dev/da1p1    /newdisk    ufs    rw    2    2
> After I entered it, I was unable to get the machine to boot Freebsd. I had to boot into safe mode. Is  This  the correct entry for etc/fstab correct?      /dev/da1p1    /newdisk    ufs    rw    2    2


Yes, that is correct. Does the mountpoint /newdisk exist? You have to create it in advance with `mkdir`. The mount will fail if it doesn't exist.
Other than that, maybe the mount failed because of the problem with the “corrupt” GPT. I don't know if it is possible to mount file systems from a GPT that is marked as “corrupt”.

Does the mount work if you type the command manually?  i.e.: `mount /dev/da1p1 /newdisk`


----------



## SirDice (Mar 5, 2019)

The Dell Precision T3600 comes with a PERC H310 or H710P SAS/SATA controller, hence the da* disks. I'm betting the disk is configured as a single disk RAID0 and the controller keeps overwriting the last blocks corrupting the gpart(8) data. Configure the controller to use JBOD.



			https://www.dell.com/downloads/global/products/precn/en/Dell-Precision-T3600-Spec-Sheet.pdf


----------



## mrredeyeflight (Mar 6, 2019)

Hello
Thank you for your help. I will look at configuring the controller to use JBOD.


----------



## mrredeyeflight (Mar 6, 2019)

It looks like the 4 TB hard disk to too big for the computer. It only can use up to a 3 TB hard drive. I will look into getting a updated Hard disk controller? I will try to contact Dell and see if they have updated Hard disk controllers?


----------



## SirDice (Mar 6, 2019)

If the disk was previously used as part of a RAID volume there may still be some metadata on the disk. Look for things like "foreign" configs, these are part of the old metadata. Clear those. Make sure the controller detects the disk as "Unconfigured Good". This means it's "clean" and ready to be configured. If the controller has any RAID functionality but you're not using it configure the disk (or the whole controller) for JBOD.


----------



## SirDice (Mar 6, 2019)

mrredeyeflight said:


> It looks like the 4 TB hard disk to too big for the computer. It only can use up to a 3 TB hard drive. I will look into getting a updated Hard disk controller? I will try to contact Dell and see if they have updated Hard disk controllers?


Most Dell PERC controllers are just relabeled LSI (now Broadcom) controllers. The PERC H310 is based on the LSISAS2008 chipset. The upper limit for SAS 1 is 2TB. This card is SAS2 and supports disks above 2TB. As far as I know a 3TB limit doesn't exist anywhere. 

I have a LSISAS2308 based card with 4 * 3TB drives and it's been running for quite some time (the volume is now 99% full, I really need to expand). As far as I know the difference between SAS2008 and SAS2308 is the PCIe version and the maximum number of devices. 


```
root@molly:~ # mpsutil show adapter
mps0 Adapter:
       Board Name: SAS9207-8i
   Board Assembly: H3-25412-00K
        Chip Name: LSISAS2308
    Chip Revision: ALL
    BIOS Revision: 7.39.00.00
Firmware Revision: 20.00.02.00
  Integrated RAID: no

PhyNum  CtlrHandle  DevHandle  Disabled  Speed   Min    Max    Device
0       0001        0009       N         6.0     1.5    6.0    SAS Initiator
1       0002        000a       N         6.0     1.5    6.0    SAS Initiator
2       0003        000b       N         6.0     1.5    6.0    SAS Initiator
3       0004        000c       N         6.0     1.5    6.0    SAS Initiator
4       0005        000d       N         6.0     1.5    6.0    SAS Initiator
5       0006        000e       N         6.0     1.5    6.0    SAS Initiator
6       0007        000f       N         6.0     1.5    6.0    SAS Initiator
7                              N                 1.5    6.0    SAS Initiator
```


```
root@molly:~ # zpool status storage
  pool: storage
 state: ONLINE
  scan: scrub repaired 0 in 1 days 12:36:51 with 0 errors on Thu Oct 19 13:48:52 2017
config:

        NAME        STATE     READ WRITE CKSUM
        storage     ONLINE       0     0     0
          raidz1-0  ONLINE       0     0     0
            da1     ONLINE       0     0     0
            da0     ONLINE       0     0     0
            da2     ONLINE       0     0     0
            da3     ONLINE       0     0     0

errors: No known data errors
```


```
root@molly:~ # smartctl -i /dev/da0
smartctl 7.0 2018-12-30 r4883 [FreeBSD 12.0-STABLE amd64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Barracuda 7200.14 (AF)
Device Model:     ST3000DM001-1ER166
Serial Number:    Z500D2BQ
LU WWN Device Id: 5 000c50 0794803f3
Firmware Version: CC25
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2, ACS-3 T13/2161-D revision 3b
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Mar  6 14:05:36 2019 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
```

If I had the money to spend I would have replaced the 3TB disks with 4 or 6 TB ones a long time ago (I don't have space for additional disks).


----------

