# Motherboards



## Terpentijn (Sep 7, 2019)

I will change from a laptop to a normal PC soon. Building one myself is no problem. However, I need some advice on a good motherboard that’s fully compatible with FreeBSD. I want ECC memory (nice for ZFS), min. 16GB but if I can afford it, I will buy 32GB ECC ram. A good LAN port is also a plus. I’ll probably will opt-in for an intel i5/i7 chip, because Ryzen is rather new and I’m not sure FreeBSD fully supports it yet. I’m not sure yet if it will be a full or mini tower. Probably the latter. Any advice is welcome.


----------



## `Orum (Sep 8, 2019)

I assume the system you are building is a desktop and not just a headless server somewhere.  FreeBSD runs just fine on Ryzen, but I've never tried running FreeBSD on their APUs or with any AMD GPUs.  Intel has other issues, as getting ECC support pretty much restricts you to server hardware, which (AFAIK) then leaves you without an iGPU.  Ryzen supports ECC out of the box (as long as your MB does too), but I'd definitely go with a dGPU then.

So I think it boils down to a few points:

Do you need ECC?  If yes (since it sounds like you do), you're probably going to have to get a dGPU no matter what.  AMD will probably be less expensive on the CPU side and give you better bang for your buck, but it may be tough to find 'desktop' oriented boards that support ECC.
If you don't need ECC, do you need a fast (i.e. discrete) GPU that works in FreeBSD?  If yes, I'd look at AMD for the CPU, and Nvidia for the GPU.  If no, Intel will probably be better as modern FreeBSD has decent support for their iGPUs.  I'm not sure about the latest Intel CPUs though, so check the supported hardware list first.
As for LAN, the Intel NICs are usually considered the crème de la crème, but as such they come at a premium.  I'd think they'd not be too hard to find on any board supporting ECC, though.  FreeBSD does support a lot of NICs (as one would expect of a server-oriented OS), and Realteks work just fine too.


----------



## rigoletto@ (Sep 8, 2019)

This all depends on the budget and details, a few examples:

Ryzen Threadripper processors are being used for quite sometime by some devs without major problems, with Gigabyte SocketTR4 mother boards.
For Intel (in general) or AMD EPIC, if fit in the budget, I would not get anything other than a Supermicro board.
For NIC Intel is always the safest choice for FreeBSD.
If you feel more adventurous you could go to Raptor Blackbird (POWER9). 

*[EDIT]*

Or, you could also get an ARMv8 Ampere (or Cavium) workstation. 
(Ampere --> PR 237055)


----------



## gpw928 (Sep 8, 2019)

Terpentijn said:


> I want ECC memory (nice for ZFS)


Intel i5,  i7, and i9 CPUs don't support ECC.
Some atom and i3 CPUs do. Verify before you buy.
AMD "consumer grade" CPUs don't disable ECC, but it's neither tested nor verified.  Be careful. I would not go this route unless you have proof it works.
EPYCs and Xeons do support ECC, but you pay for it.
I would not buy a CPU or motherboard without VT-x, VT-d, and AES-NI (or AMD equivalent) support.  The whole world is going virtual, so you might as well be prepared.
If I were building a ZFS server again, I would make sure I had an NVMe disk (mirror) for the ZIL (and probably the root).  It would not need to be large, but would have to have "power loss protection" (usually referred to as "enterprise class" by the marketing people who love to obfuscate).
If you are thinking about RAID-Z for the "tank", think again and go RAID-Z2 (I wish I had).


----------



## Terpentijn (Sep 8, 2019)

Thank you. ECC was mainly an option to make ZFS more failure proof. From the info above I get the impression that ‘power loss protection’ gives me enough safety. I’m not a gamer, so a dGPU is not really needed. Intel will do just fine.


----------



## Terpentijn (Sep 8, 2019)

gpw928 said:


> Intel i5,  i7, and i9 CPUs don't support ECC.
> Some atom and i3 CPUs do. Verify before you buy.
> AMD "consumer grade" CPUs don't disable ECC, but it's neither tested nor verified.  Be careful. I would not go this route unless you have proof it works.
> EPYCs and Xeons do support ECC, but you pay for it.
> ...


I was indeed thinking about RAID-Z for the (data) ‘tank’. It needs at least 3 disks. Why do you wish you had chosen RAID-Z2? It needs at least four disks. So, if I install 4x8TB drives, I still have only 8TB storage? It’s a personal desktop computer. I want my data to be ‘safe’ but Z2 is overkill imho. Mirrored disks for home use are enough. I’ll just make sure I have backups of important data  Do I really need a ZIL in a home desktop computer?


----------



## `Orum (Sep 8, 2019)

If you're just doing a pool of three disks, and you're not storing ultra-critical data or you do regular backups, RAID-Z1 is fine.  It's mainly on large vdevs that you want to consider -Z2.  I try to keep redundancy at no less than 25% of disks in a vdev as parity disks, i.e. use -Z1 for up to 4 disks, and -Z2 for 5 to 8.  I never put more than 8 in a vdev, as the cost to IOPS doesn't make out.

Of course with drives getting larger and larger, and read/write speeds (per TB) not keeping pace, things can get dicey with high density drives.  It takes longer and longer to resilver, and drive failures are correlated, so it might make sense to use even more parity now.  Once multi-actuator drives become commonplace this should be less of an issue, but we're not there yet.


----------



## Terpentijn (Sep 8, 2019)

According to benchmarks the Threadripper is better for workstation / Ryzen is the desktop processor. The new pc will mainly be for desktop use. No games. Photo and video editing and the usual stuff. So I guess the Ryzen is a better choice.

I selected some parts which seem to be good.

```
CPU: AMD Ryzen 7 3800X
GPU: Nvidia GTX 1660-Ti
RAM: Corsair Vengeance LPX DDR4 3200 C16 2x16GB
MBD: Gigabyte X570 AORUS ELITE
```

I’ll start with my old SSD’s and WD drives

*Still looking for a nice case and a future proof power supply*.
If possible the system needs to be silent.

Any good advice on the case/ power?

AFAIK the selected parts are fully compatible with FreeBSD, based on information on the net and this forum. However, the Ryzen is not mentioned in the official FreeBSD list. I don’t know why. So, if any info exists on not supporting the chosen parts I would like to know


----------



## rigoletto@ (Sep 8, 2019)

Terpentijn said:


> *future proof power supply*.
> If possible the system needs to be silent.



Seasonic make some of the best and most stable PSU for desktop, and there are fanless models.


----------



## CraigHB (Sep 8, 2019)

Terpentijn said:


> I will change from a laptop to a normal PC soon



I retired my desktop systems and went to a laptop after moving to a smaller place.  I've discovered I really miss my desktop system.  I'm in the same boat, looking to put one together and of course run FreeBSD on it.  I'm not looking for ECC though, narrows the field too much in terms of component selection.  

Don't have any specific recommends, but I've always been happy with Gigabyte products and I'll be using one of their Aorus boards.  I'm actually planning to use a Ryzen CPU.  I've not read of any support issues with FreeBSD.  The AMD stuff is pulling ahead of Intel in several ways, cost being a big one.


----------



## rigoletto@ (Sep 8, 2019)

If you are looking for compact but serious system you may want to have a look on THIS one.

*[EDIT]*

Btw, the Mailing Lists and IRC channels are better channels to get info about technical specialties specifities.


----------



## CraigHB (Sep 8, 2019)

I was actually thinking in terms of mini-ITX, but there are some limitations there.  I think what I'll do to save space is use a proper desktop case as in the under the monitor kind.  I do plan to build another system with a mini-ITX case, but that would be for a home theater PC.


Terpentijn said:


> If possible the system needs to be silent.



Yeah fans are kind of the noise pollution scourge of desktop computers.  My last one had only a 120mm power supply fan along with the usual CPU and GPU fans.  It seemed to stay cool enough and made zero noise at idle, though it could make some fan noise when working hard.  Unfortunately the mainboard I'm looking at has a chipset fan which I'm not happy about.  If it makes noise I'll have to deal with it.


----------



## Terpentijn (Sep 8, 2019)

rigoletto@ said:


> If you are looking for compact but serious system you may want to have a look on THIS one.
> 
> *[EDIT]*
> 
> Btw, the Mailing Lists and IRC channels are better channels to get info about technical specialties.


You’re right. I’ll close this thread. Power supplies and cases are not FreeBSD related anyway.


----------



## rigoletto@ (Sep 8, 2019)

Terpentijn said:


> Power supplies and cases are not FreeBSD related anyway.



No problem talking about them in here.


----------



## Terpentijn (Sep 8, 2019)

CraigHB said:


> I retired my desktop systems and went to a laptop after moving to a smaller place.  I've discovered I really miss my desktop system.  I'm in the same boat, looking to put one together and of course run FreeBSD on it.  I'm not looking for ECC though, narrows the field too much in terms of component selection.
> 
> Don't have any specific recommends, but I've always been happy with Gigabyte products and I'll be using one of their Aorus boards.  I'm actually planning to use a Ryzen CPU.  I've not read of any support issues with FreeBSD.  The AMD stuff is pulling ahead of Intel in several ways, cost being a big one.


I’ve used my laptop for years but I need more power. To buy a laptop with the power I want would cost a small fortune. So, I’m going back to a normal tower pc. Far less money. Far more power.


----------



## rigoletto@ (Sep 8, 2019)

You may want to have a look on Streacom and HDPlex fanless chassis; however they are at the expensive side.


----------



## CraigHB (Sep 8, 2019)

Terpentijn said:


> I’ve used my laptop for years but I need more power.



Yeah that's what I miss about my desktop system, also the flexibility, more ports, more options.  The laptop I have is pretty good in terms of power, but a desktop for the same amount of money would have a lot more power. 

I do a lot of Blu-Ray rips for my home media system.  Video encoding is really power hungry and time consuming.  An unthrottled 16 thread CPU over the current 12 thread throttled CPU of my laptop would provide a big time savings for me.


----------



## gpw928 (Sep 9, 2019)

Terpentijn said:


> I was indeed thinking about RAID-Z for the (data) ‘tank’. It needs at least 3 disks. Why do you wish you had chosen RAID-Z2? It needs at least four disks. So, if I install 4x8TB drives, I still have only 8TB storage? It’s a personal desktop computer. I want my data to be ‘safe’ but Z2 is overkill imho. Mirrored disks for home use are enough. I’ll just make sure I have backups of important data  Do I really need a ZIL in a home desktop computer?


4 x 8 TB = 32 TB.  You lose one disk's worth (8 TB) for RAID-Z1, and two disk's worth (16 TB) for RAID-Z2.  So you would have 24 TB and 16 TB respectively for RAID-Z1 and RAID-Z2.

It all comes down to the risk profile you are willing to accept.  

For decades, my home computers had no RAID.  I survived.  But I also had backups.

The last time one of my tank disks died, I knew what would happen if I pulled the wrong disk to replace.  Two dead disks in RAID-Z1 means total loss.  I spent a lot of time identifying the dead disk.  Same problem with a degraded mirror.

My view is that disks are cheap enough that I want maximum redundancy.

I believe that the ZIL will be in the tank by default.  The value of moving it to dedicated fast storage would depend on your load.  If it's modest, you are correct.

However it's worth mentioning that NVMe provide a whole new experience in disk speed.  I would want some of that in the mix...


----------



## Phishfry (Sep 9, 2019)

For any consideration of motherboards the essential question is; How many PCI/PCIe slots will you need.
This will help determine the size of the board that best fits your needs.
ITX is limited to a single PCI/PCIe slot. So what peripherals will you be using?
MATX is usually enough for most tasks with 2 slots at x8 lanes(PCIe 3.x) and one slot from chipset(usually PCIe 2.x).

So  how many PCIe 3.x lanes do you need. If more than 16 than consider LGA2011 or the newer LGA3647.
If less than 16 lanes PCIe 3.x are needed than consider LGA1151.
As for manufacturers I support SuperMicro. They are not perfect but offer a great set of features.
With all the Spectre vulnerabilities going around one important consideration is frequency of BIOS updates.

I used to recommend Gigabyte.
But I recently had a clients machine on the bench the other day and went to update the BIOS and noticed that Gigabyte had not released any BIOS updates beyond the first version. So SuperMicro might cost more but they have support.


----------



## gpw928 (Sep 9, 2019)

Regarding RAID-Z2, I should have added that ZFS tanks are often too big to back up 100%.  Mine is.

I have about 800 GB of "high value" things quarantined for routine off-site backup.  The rest is "at risk".

RAID-Z2 is the risk mitigation I wish I had.


----------



## CraigHB (Sep 9, 2019)

gpw928 said:


> However it's worth mentioning that NVMe provide a whole new experience in disk speed. I would want some of that in the mix...



Yeah NVMe is amazing.  My laptop computer has a fast Samsung 3.0x4 drive and it benchmarks at crazy speeds, over 3 GBytes/s on both sequential read and write.  Though it really shines on large file transfers.  For things like system boot and loading apps it doesn't make as much of a difference over SATA, but the speeds are definitely impressive and really nice to have when handling large files.

Speeds are even more crazy with the latest drives hot off the press.  With PCIe version 4.0 made available with AMD's 3rd gen CPUs and chipsets you can get even more speed out of NVMe.  The latest PCIe 4.0 drives are seeing sequential speeds as high as 5 GBytes/s, insane.  Put ~that~ in your RAID array and smoke it.

My one complaint about NVMe is they've made the M.2 form factor the consumer standard.  U.2 is actually better for me since I like to use removable disks and it's greatly better for disk arrays.  However U.2 didn't really get off the ground like it should have.  U.2 disks are not nearly as abundant in terms of product range and they're a lot more expensive.  M.2 to U.2 disk adapters are available and inexpensive, but no consumer motherboards have U.2 connectors on them.


----------

