# Intel Xeon vs. Ryzen



## stratacast1 (Jul 27, 2019)

Hey guys,

I'm not here to start a flame war, just trying to get some opinions on my upgrade path. I upgraded my desktop with the intent of switching my Ryzen 1600 and its mobo to be a server. However, despite Asus claiming the board supports ECC, they have let me down with false information. So I'm in a pickle. I could ask the community if they have a Ryzen server with ECC and what board they use and I'll consider throwing some extra $$ down if it means I can get that working. Otherwise...what are my prospects with a Xeon that supports DDR4? I'd really like to go used if possible to keep costs down. How would a 4C Xeon from the last few years fare against a Ryzen chip with performance and stability for FreeBSD with bhyve and jails? I don't want to dip into DDR3 gens because I have 32GB of DDR4 ECC ready to go.


----------



## Remington (Jul 27, 2019)

AFAIK there isn't a server grade motherboard that currently supports Ryzen processor.  If you are looking for server grade motherboard and I would suggest you to look into Supermicro or Tyan motherboards with Xeon processors since they're designed to run 24/7 for 5 to 7 years.  ASUS is more of a desktop motherboard manufacturer catering to high end gamers and I wouldn't use them as server.

Your best place is to look for used Supermicro or Tyan motherboard on ebay or something similar.  These still work very good and many companies retire them after 3 years to avoid downtime when the server or hard drive unexpectedly fails.  You get a bargain price for used server.

I have 2 servers running 24/7 for 5 years without any problems.  It's time for me to make plans to replace them or keep it as a backup server.


----------



## stratacast1 (Jul 28, 2019)

Remington said:


> AFAIK there isn't a server grade motherboard that currently supports Ryzen processor.  If you are looking for server grade motherboard and I would suggest you to look into Supermicro or Tyan motherboards with Xeon processors since they're designed to run 24/7 for 5 to 7 years.  ASUS is more of a desktop motherboard manufacturer catering to high end gamers and I wouldn't use them as server.
> 
> Your best place is to look for used Supermicro or Tyan motherboard on ebay or something similar.  These still work very good and many companies retire them after 3 years to avoid downtime when the server or hard drive unexpectedly fails.  You get a bargain price for used server.
> 
> I have 2 servers running 24/7 for 5 years without any problems.  It's time for me to make plans to replace them or keep it as a backup server.



Yeah that's another reason why I'm reconsidering and checking out Xeons again. Part of me trusts the longevity of my motherboard but maybe I'm just fooling myself. I have an ancient X3430 that's gotta go. It's been running great in my possession for 3 years now. Just really wish I could go Ryzen and have something different than Intel parts


----------



## Remington (Jul 28, 2019)

If you're looking for something other than Intel.  I have 2 AMD Opteron with total of 32 cores in one of my server with Tyan motherboard.  Epyc is the current successor.


----------



## stratacast1 (Jul 28, 2019)

I would probably end up with EPYC, but that's going a bit extreme, this isn't for anything that requires that much power...so if I want anything my option is sadly Xeon...wish AMD would try to fill this hole in the market. 

This could be a good option








						HP Z440 Workstation Intel Xeon Quad Core 3.5GHz 8GB DDR4 500GB HD NVIDIA Win 10  | eBay
					

Find many great new & used options and get the best deals for HP Z440 Workstation Intel Xeon Quad Core 3.5GHz 8GB DDR4 500GB HD NVIDIA Win 10 at the best online prices at eBay! Free shipping for many products!



					www.ebay.com
				




My current box is an HP Proliant and its been very good to me. I'd just swap in my existing drives and memory and plug along since it also supports DDR4


----------



## Remington (Jul 28, 2019)

stratacast1 said:


> I would probably end up with EPYC, but that's going a bit extreme, this isn't for anything that requires that much power...so if I want anything my option is sadly Xeon...wish AMD would try to fill this hole in the market.
> 
> This could be a good option
> 
> ...



It looks good.


----------



## stratacast1 (Jul 28, 2019)

Might just do that then...kinda defeated the purpose of a few pieces of hardware I purchased but maybe I can piece it all together and make back a few hundred on the deal...


----------



## toorski (Jul 28, 2019)

*Purchased used from craigslist  for $300 - 2 years ago*
This is my email server (laptop with no-ECC RAM) - hosting smtpd and imap daemons in jail.








```
FLOPS C Program (Double Precision), V2.0 18 Dec 1992

   Module     Error        RunTime      MFLOPS
                            (usec)
     1      4.0146e-13      0.0067   2093.5276
     2     -1.4166e-13      0.0058   1206.1841
     3      4.7184e-14      0.0048   3507.6718
     4     -1.2557e-13      0.0047   3207.7199
     5     -1.3800e-13      0.0085   3417.3477
     6      3.2380e-13      0.0072   4014.9609
     7     -8.4583e-11      0.0134    897.2027
     8      3.4867e-13      0.0098   3059.2490

   Iterations      =  512000000
   NullTime (usec) =     0.0000
   MFLOPS(1)       =  1535.5699
   MFLOPS(2)       =  1779.4692
   MFLOPS(3)       =  2649.7111
   MFLOPS(4)       =  3427.2306
```

Sometimes I access the laptop over xrdp with freerdp to look at things in Lumina DE there, if I rly need to see it in GUI. *On-24/7ops.

Purchased used from craigslist for $250 -2 years ago*
This is my Xeon server/workstation (ECC RAM)  - hosting authoritative master and cashing dns, smtpd, imap, few httpd(s), pgsql, php, phython, java and js dev. frameworks, plus more that I can’t think of ATM, all in various jails. Here, I sit directly front of Lumina DE to experiment, learn and play FreeBSD in GUI. *On-24/7 ops.*








```
FLOPS C Program (Double Precision), V2.0 18 Dec 1992

   Module     Error        RunTime      MFLOPS
                            (usec)
     1      4.0146e-13      0.0075   1859.7054
     2     -1.4166e-13      0.0065   1076.6817
     3      4.7184e-14      0.0058   2922.2714
     4     -1.2557e-13      0.0062   2435.3352
     5     -1.3800e-13      0.0089   3259.5171
     6      3.2380e-13      0.0088   3291.2893
     7     -8.4583e-11      0.0151    797.0276
     8      3.4867e-13      0.0120   2504.9222

   Iterations      =  512000000
   NullTime (usec) =     0.0000
   MFLOPS(1)       =  1356.8281
   MFLOPS(2)       =  1559.9871
   MFLOPS(3)       =  2272.5397
   MFLOPS(4)       =  2777.4161
```


Bottom line, I don’t see much of performance or reliability difference between those 2 systems. As you can see, according to *“flops”* (floating point operations per second), the laptop’s performance is better than Xeon’s

I’ll build new system with *Ryzen* for FreeBSD, when I get all the components. I should have all in by the end of this week ready and running FreeBSD – I hope. You can look in there to see my progress with Ryzen and FreeBSD.








						My wish list of hardware components for new FreeBSD-12.* build.
					

I have few hours to decide if those are the best and ultimate components for a new build that I plan to house fresh FreeBSD-12.0 server/workstation in.  https://www.newegg.com/p/N82E16811353062?Item=N82E16811353062 https://www.newegg.com/asus-tuf-b450m-plus-gaming/p/N82E16813119141...




					forums.freebsd.org


----------



## stratacast1 (Jul 28, 2019)

I've been going over the limits on my box. Nextcloud operations on lots of files or compression/decompression and such take a long time (loooong time), I'm streaming with Plex, backups to the disks, web servers, ssh proxy, other file storage/syncing, and I plan on moving my VPN over to it so I can turn one of my boxes into something else...in addition to some other projects that are in the works that will be set up to run on this.

My Xeon is an X3430 and I can't even utilize my SSDs since everything is SATA II. There is a big difference in performance with what I do even between boxes 3 years older than what I have. I know from having worked with them. My demands have grown from it being my own little thing to sharing with more folks. Reliability isn't my issue, this 10 year old box is ol' faithful. But I need more horsepower


----------



## Remington (Jul 28, 2019)

toorski said:


> I’ll build new system with *Ryzen* for FreeBSD, when I get all the components. I should have all in by the end of this week ready and running FreeBSD – I hope. You can look in there to see my progress with Ryzen and FreeBSD.
> 
> 
> 
> ...



That's kinda overkill for a server and it doesn't support ECC so that's a no-no.  It might be okay for experiment or desktop uses but in the long run data will become corrupted if run 24/7.  Is that something you can afford to lose?


----------



## Phishfry (Jul 28, 2019)

I say buy a new SuperMicro X10 board that will take your DDR4 and buy a used V3 LGA2011.(V4 chips have held their price)








						Supermicro X10SRL-F Motherboard S-2011 for E5-2600/1600 v3
					

Motherboard ATX Intel C612 Express Chipset Socket R3 (LGA 2011) for Single Intel Xeon E5-2600/1600 v3 family processors - Supports up to 512GB DDR4 ECC LRDIMM (or) up to 256GB DDR4 ECC RDIMM (8x 288-pin DDR4 DIMM sockets) - On-board : SATA3 (6Gbps) via C612 controller (RAID),Integrated IPMI 2.0...




					www.wiredzone.com
				











						Intel Xeon E5-2608L V3 SR21P 2.00GHZ CPU *km  | eBay
					

Find many great new & used options and get the best deals for Intel Xeon E5-2608L V3 SR21P 2.00GHZ CPU *km at the best online prices at eBay! Free shipping for many products!



					www.ebay.com


----------



## Phishfry (Jul 28, 2019)

I really prefer the LV (Low Voltge) suffixed LGA2011 chips.
They have been around since LGA2011 V1.
The same chip monikers are used among all 4 generations.
For example 2630LV* has been a low core count with higher clock speed than most LV chips.
The Mac daddy is 2650LV* with the most cores in a low power envelope. They do hold thier value.
I have low core count 2608LV3 and they are pretty affordable.
The 16xx series was very unimpressive to me unless high clock speed is needed for single threaded apps.








						Intel Xeon 8 Core E5-2618LV3 SR200 2.3GHZ 20MB 8 GT/S LGA 2011 V3 CPU Processor  | eBay
					

Find many great new & used options and get the best deals for Intel Xeon 8 Core E5-2618LV3 SR200 2.3GHZ 20MB 8 GT/S LGA 2011 V3 CPU Processor at the best online prices at eBay! Free shipping for many products!



					www.ebay.com
				



To me I don't like buying used motherboards because the sockets are so flimsy. One wrong move and it is dead.
Too many ham fisted ebayer's out there. Repairing sockets fingers is no joke. You better have a nice magnifier.
.
LV Cheat sheet
2630L=Sandy Bridge
2630LV2=IvyBridge
2630LV3=Haswell
2630LV4=Broadwell

SuperMicro boards:
V1 and V2 chips will all work in same board (C602 Chipset)
V3 and V4 chips will all work in the same board (C612 Chipset)

The problem you find with new boards is sometimes they ship with older BIOS that only supports V1 CPU but will support V2 CPU's with a BIOS update. Same with V3 boards.
Stale motherboard inventory may mean you need a V3 CPU to upgrade the BIOS to take V4 chips.
You will see many NewEgg complaints that 'board does not work' when they are not flashed with newer BIOS.
The SuperMicro boards have the ability to upgrade the BIOS without a CPU via the BMC webpage but it is an add-on option that SuperMicro wants to charge you for. Does not work without a serial number. Bahhhh to them.


----------



## Phishfry (Jul 28, 2019)

Let me give the skinny on HP Xeon boxes.
Motherboard uses custom 18 pin power supply header and is non-standard gear.
I think you can do better.

Notice the connectors from this pic:








						HP Z440 Workstation Motherboard LGA 2011 Ddr4 710324-002 761514-0 for sale online | eBay
					

Find many great new & used options and get the best deals for HP Z440 Workstation Motherboard LGA 2011 Ddr4 710324-002 761514-0 at the best online prices at eBay! Free shipping for many products!



					www.ebay.com


----------



## Phishfry (Jul 28, 2019)

I think you can add the Intel S2600 series motherboards in your search too.
The part numbers are screwy so you need to look hard at the details.

Been wanting to build a ITX virt hotrod but I am all in on EATX these days.





						ASRock Rack EPC612D4I Mini ITX Server Motherboard LGA 2011 R3 Intel C612 for sale online | eBay
					

Find many great new & used options and get the best deals for ASRock Rack EPC612D4I Mini ITX Server Motherboard LGA 2011 R3 Intel C612 at the best online prices at eBay! Free shipping for many products!



					www.ebay.com
				




Would you trust Asrock Rack (their server brand) in a 24/7 rig?


----------



## Phishfry (Jul 28, 2019)

The Intel E5-2695 and 2697 series are the real fast ones with lots of cores. They do burn around 135 watts.
Which is somewhat linear. The LV chips burn around 70W and have around half the benchmarking performance.


----------



## Remington (Jul 28, 2019)

Phishfry said:


> Would you trust Asrock Rack (their server brand) in a 24/7 rig?



I wouldn't trust ASUS or Asrock as a primary server.  Supermicro and Tyan motherboards have excellent track records of being reliable 24/7.  Also Intel have excellent server grade motherboards but its more expensive.


----------



## toorski (Jul 28, 2019)

Phishfry said:


> The problem you find with new boards is sometimes they ship with older BIOS that only supports V1 CPU but will support V2 CPU's with a BIOS update. Same with V3 boards.
> Stale motherboard inventory may mean you need a V3 CPU to upgrade the BIOS to take V4 chips.
> You will see many NewEgg complaints that 'board does not work' when they are not flashed with newer BIOS.
> The SuperMicro boards have the ability to upgrade the BIOS without a CPU via the BMC webpage but it is an add-on option that SuperMicro wants to charge you for. Does not work without a serial number. Bahhhh to them.


looks like, one can grab new SuperMicro X10 for a reasonable price. BUT then, how much do they want for upgrade of the BIOS - more than selling price of the motherboard?  They always have a hook - payless now so we can charge  you more later  

And then, one has to deal with Ebay sellers, and their game of no shame, to get rest of the so called server grade components, for cheap, that supposedly worked until they arrive at buyer's location. Then, they don't work, are not as described or worse. I gave that game up, along with PayPal, long time ago.


----------



## Phishfry (Jul 28, 2019)

BIOS downloads are free. It is only this silly 'Broadband Managment Controller' webpage that has an option to flash via BMC.
That will allow a BIOS flash with no CPU installed.
It is some sort of BMC option with serial number needed.

All the normal flashing tools still work fine. I still keep an MS-DOS memstick for flashing.
Supermicro also does a good job of supporting BIOS upgrades.
I just got a fresh BIOS for my X9SRL from Nov2018. Not bad for a 7 year old motherboard.
They also keep their BMC controller updated.


----------



## Remington (Jul 28, 2019)

toorski said:


> And then, one has to deal with Ebay sellers, and their game of no shame, to get rest of the so called server grade components, for cheap, that supposedly worked until they arrive at buyer's location. Then, they don't work, are not as described or worse. I gave that game up, along with PayPal, long time ago.



Ebay and PayPal have toughen up a lot lately protecting the buyers from scammers.  If you receive anything wrong or item not as described then you send it back and file a complaint with ebay/paypal to get your money back.  I have done it once recently and it was resolved quickly.


----------



## Phishfry (Jul 28, 2019)

Remington said:


> I wouldn't trust ASUS or Asrock as a primary server.


Asus does seem to have a couple of server boards but they use custom backplane risers for PCIe.
I think they are rack boards. I am not very interested in them either. P5-B was a great board!

I have a Gigabyte server line board LGA1151 with E3-1240v6 and it works fine, but their BIOS updates are not nearly as timely as SM.
The Gigabyte LGA2011 boards are not very popular but they are OEM'd to some other brands I have noticed.
Most notably Datto.

If you are going to use bhyve you want cores and memory. Your 32GB will be paltry with a couple of VM's. No offense.
I do feel E5 Xeons are better suited to bhyve than E3 Xeons. Did I mention you want lots of cores?


----------



## Phishfry (Jul 28, 2019)

toorski said:


> looks like, one can grab new SuperMicro X10 for a reasonable price.


Yes the X10 title can be deceiving as well. They have X10 boards with LGA1151 and  LGA2011
Same with the X9 Line. They had LGA1150, LGA1151, LGA1356 and LGA2011.

The real key to their system is the next set of letters.

D= Dual Socket and S=Single
X9S** is single socket Sandy/Ivybridge
X10S** is single socket Haswell/Broadwell

The next digit in SuperMicro model numbers is the series. So LGA2011 has a digit for the series. it is 'R'
X9SR* is single socket LGA2011
X10SR* is single socket LGA2011-3
When it comes to dual socket boards they add a letter 'A'
X9DA* is a dual socket LGA2011
X10DA* is a dual socket LGA2011-3
X9DR* is also a dual socket LGA2011
X10DR* is also dual socket LGA2011-3

Then comes the specialty of the board (I have not fully figured these out)
L = For example X9SRL and X10SRL
I = For example X9SRI and X10SRI <<< These seem to have iSCSI booting in the BIOS where others do not.
A = For example X9SRA and X10SRA <<< This is definitely the audio option board.
H = For example X9DRH and X10DRH <<< This option adds onboard SAS controller
G = For example X9DRG and X10DRG <<< These are optimized for graphics cards with 4X dedicated slots full bandwidth for GPU


----------



## steveoc64 (Jul 29, 2019)

I thought AsrockRack had a "server grade AM4" board in their catalog









						AsRock Rack X470D4U Micro ATX Server Motherboard AM4 Ryzen - Newegg.com
					

Buy AsRock Rack X470D4U Micro ATX Server Motherboard AM4 Ryzen & Ryzen 7nm PGA1331 AMD X470 with fast shipping and top-rated customer service. Once you know, you Newegg!




					www.newegg.com
				




Might be worth a look, esp if you already have some AM4 CPUs floating around ?

My (short) experience with FreeBSD on Ryzen is a bit of a mixed bag.
- Getting so much processing power and cores, for the money, and relatively low power consumption  == good 
- Getting bhyve failing catastrophically trying to boot - Ubuntu, Dragonfly, Win10 == not so good
- I have no problems on this machine loading FreeBSD in a bhyve under freebsd mind you.
- Support the market disrupter and all that == good

Ive had no problems getting bhyve running on i3 / i5 / Xeon 1151 ... but zero luck with Ryzen so far.  Hopefully its a bad batch of hardware ?


----------



## rigoletto@ (Jul 29, 2019)

That probably is a board where you can use ECC memory but without actual ECC support.

You could have a look on some Cavium ThunderX (v1), and FreeBSD is supposedly (I don't have one to assure) to be quite stable on ARMv8. Also, IIRC, the ThunderX (V1) isn't affected by Spectre. Or the Raptor Blackbird (POWER9), not so stable I think but should be OK for a home server - and fun.

*[EDIT]*

Some FreeBSD dev (I don't remember who) got one of those dual socket Raptor and is running FreeBSD on it.

*[EDIT]*

Oh, there is Ampere eMAG ARMv8 processors too. See PR 237055.


----------



## Phishfry (Jul 29, 2019)

Here is the Asus AMD server board I was referencing above.
The ONLY reason I considered it, was  because it one of the only serverboards to take coreboot.
G34 socket is unknown to me. Throw in those funky PCIe sockets which need a custom riser and I was lost.








						ASUS KGPE-D16, Socket G34, AMD (90-MSVD01-G0UAY00Z) Motherboard for sale online | eBay
					

Find many great new & used options and get the best deals for ASUS KGPE-D16, Socket G34, AMD (90-MSVD01-G0UAY00Z) Motherboard at the best online prices at eBay! Free shipping for many products!



					www.ebay.com
				



I think system76 used these in their open source hardware rack offering. Was pretty darn expensive too.



rigoletto@ said:


> That probably is a board where you can use ECC memory but without actual ECC support.


I see that in some of my hardware. APU1/2/3 uses ECC soldered onboard but problems implementing it with coreboot.
Probably some patented alogos used...

I bought a whole bunch of DDR3 4GB ECC modules for bulk price and was surprised that they worked on some desktop boards.


----------



## steveoc64 (Jul 29, 2019)

Socket G34 are awesome !

Those old 12 / 16 core opterons are going for peanuts on ebay still, and they (should) perform per-core on a par with Bulldozer (which is pretty good)

Quite a few of those off-brand Chinese board makers are offering brand new G34 boards (with custom heatsink) for under the $100 mark.  Been tempted to try one just for fun.


----------



## Phishfry (Jul 29, 2019)

steveoc64 said:


> Quite a few of those off-brand Chinese board makers are offering brand new G34 boards (with custom heatsink) for under the $100 mark. Been tempted to try one just for fun.


Are these the boards I see labeled as X89 chipset??? Cracks me up they are copying Intel monikers. (ie. X79 X99)
just like the Asrock Rack board above. X470 makes it sound like an Intel chipset. (ie. X270, X370)

I found the Opteron 6380 has a 10K passmark score and that is good for a 7 year old board.
The Opteron 6370p sounds more my style at 99W TDP instead of 115W on 6380.
It is a rare CPU and appeared to be a fresher version using Warsaw.

G34 Drawbacks, DDR2 and SATA2, biggest hit for me is PCIe 2.x on those chips boards.
The SuperMicro H8 boards look good but still not cheap.

Overall I am happy I ended up on LGA2011.
It is pretty versatile socket and lots of cheap cooling as well as heatpipes and copper.

LGA3647 and EPYC sound groovy but I do have a house payment to make.


----------



## steveoc64 (Jul 29, 2019)

yep, thats the one.

says DDR3 on the specs though, ecc supported, apparently.

or go supermicro for 200, German supplier, ipmi and all the good stuff.


----------



## Phishfry (Jul 29, 2019)

The SM Opteron boards do seem to mirror thier Intel conterparts.
How about an EPYC H11SSL.
128 Lanes of PCIe bus and the clowns put x8 slots on the board.
All need to be 16X you idiots. Who designs this stuff. Geez.
They act like they never designed a server board.
Why hold back. Are X16 slot parts cost really that much higher? Maybe so many traces they need more PCB layers?
Inquiring minds want to know.....

Plus what is up with this:


> Up to 32 Cores


i thought EPYC was carrying lots of cores too. 32 sounds low these days.....
Intel is packing more in their LGA3647 GOLD line. But you do need to be somewhat wealthy to partake.


----------



## toorski (Jul 29, 2019)

Phishfry said:


> Why hold back. Are X16 slot parts cost really that much higher? Maybe so many traces they need more PCB layers?


They'll loose all the FreeBSD gamerz, as potential customers, without  x16 PCIe slot on that motherboard


----------



## Phishfry (Jul 29, 2019)

EPYC chips do top out at 32 cores 64 threads. (For Now)


			https://www.amd.com/en/products/epyc-7000-series-1-socket-models?items_per_page=6&page=1
		


No EATX Single socket boards.





						Motherboards | Supermicro
					

Supermicro’s in-house design superiority is tailored for the needs of gaming, IoT or Enterprise; consistent highest-quality and production expertise




					www.supermicro.com
				



The H11DSi has 256 Lanes of PCIe available yet they thought it was ok to use some x8 slots in there.


----------



## Phishfry (Jul 29, 2019)

Maybe it is all a big conspiracy. To get early access to Intel silicon SM must behave.
They use this A+Plus division and sell neutered AMD gear with their overlords permission.
So then Intel also gets early access to AMD samples via them.


----------



## Remington (Jul 29, 2019)

Phishfry said:


> i thought EPYC was carrying lots of cores too. 32 sounds low these days.....
> Intel is packing more in their LGA3647 GOLD line. But you do need to be somewhat wealthy to partake.



You can add more cores by finding motherboard that can support 2 or 4 epyc chips.  It's definitely overkill for what you need.


----------



## olli@ (Jul 29, 2019)

Phishfry said:


> i thought EPYC was carrying lots of cores too. 32 sounds low these days.....


Well, 32 cores (64 threads) isn't too bad for a single package. There are servers that take multiple packages. For example, the ASUS RS700-E9-RS12 has two Epyc CPU sockets, so you have 64 cores (128 threads) in a single system. It supports ECC, by the way (up to 2 TB octa-channel DDR4).

The upcoming Ryzen3-based Epyc is rumored to have up to 48 cores (96 threads), coming close to 200 threads for a two-package system. That's pretty good for a single x86-based node. And if that's still not enough for you, well, then just put more of those servers into your rack.


----------



## Phishfry (Jul 29, 2019)

Yes you are correct plus you need to go to Intel LGA3647 'PLATINUM' for over 24 cores on Intel. GOLD maxes at 24 Cores.
PLATINUM does go up to 56 cores.

It does only offer 6 channel RAM versus 8 channel on EPYC.


----------



## stratacast1 (Jul 31, 2019)

Phishfry said:


> Let me give the skinny on HP Xeon boxes.
> Motherboard uses custom 18 pin power supply header and is non-standard gear.
> I think you can do better.
> 
> ...



I wouldn't have caught that, thank you for saving me some headache down the road! I think I can afford that new, shiny board...at least the CPUs are cheap



Phishfry said:


> If you are going to use bhyve you want cores and memory. Your 32GB will be paltry with a couple of VM's. No offense.



With what I have going on, 32GB is more than plenty. But it seems a lot of these boards are quad channel(?) so if I ever want an upgrade I can just get the same set of sticks again and go up to 64GB. Right now my VM projects are on low-duty because I only have 8GB on what I'm using...and my real server has 16GB, and I plan on moving those VMs over to the new server and really start utilizing my VMs more than just being plinkers.




steveoc64 said:


> I thought AsrockRack had a "server grade AM4" board in their catalog
> 
> 
> 
> ...



I do have a Ryzen 1600 I was wanting to use...I upgraded my tower under the impression it wouldn't be a big deal going Ryzen for my server since my current server is ancient and slow. But, I use bhyve and want to use it more and the fact that Ryzen + bhyve don't get along well is a dealbreaker for me. You're the second person I've heard having difficulties with Ryzen and bhyve.




rigoletto@ said:


> You could have a look on some Cavium ThunderX (v1), and FreeBSD is supposedly (I don't have one to assure) to be quite stable on ARMv8.



That would be awesome if I could just get the board/CPU for a good price and not have to shell out for a whole box!


----------



## rigoletto@ (Jul 31, 2019)

stratacast1 said:


> That would be awesome if I could just get the board/CPU for a good price and not have to shell out for a whole box!



You may try to talk with Gigabyte or if you are located in Europe with Avantek; however I could ask a friend in the project who has a foot in the ARM _métier_ if he can help somehow. In the last case you can ping me in private; however I think that would be difficult to happen. 

This one seems to be in USA.


----------



## steveoc64 (Jul 31, 2019)

Gotta say - those Raptor Blackbird / Power 9 boards are looking interesting.

I know its way off the mark from what the OP was asking ... but still


----------



## olli@ (Jul 31, 2019)

stratacast1 said:


> I do have a Ryzen 1600 I was wanting to use...I upgraded my tower under the impression it wouldn't be a big deal going Ryzen for my server since my current server is ancient and slow. But, I use bhyve and want to use it more and the fact that Ryzen + bhyve don't get along well is a dealbreaker for me. You're the second person I've heard having difficulties with Ryzen and bhyve.


Just for the record … I've got a Ryzen 2700 and tryed bhyve for a quick test. It worked well in general, but bhyve doesn't support passing a Blu-Ray drive to a guest, so I had to switch to VirtualBox which supports that. VirtualBox works very well with the Ryzen.


----------



## steveoc64 (Jul 31, 2019)

olli@ said:


> Just for the record … I've got a Ryzen 2700 and tryed bhyve for a quick test. It worked well in general, but bhyve doesn't support passing a Blu-Ray drive to a guest, so I had to switch to VirtualBox which supports that. VirtualBox works very well with the Ryzen.


I cant get bhyve working on ryzen with ubuntu. .. several versions.

wonder what im not setting up?

any magic tricks you can think of?


----------



## olli@ (Jul 31, 2019)

steveoc64 said:


> I cant get bhyve working on ryzen with ubuntu. .. several versions.
> 
> wonder what im not setting up?
> 
> any magic tricks you can think of?


No, I didn't do anything special, as far as I can remember. I installed Suse, though, not Ubuntu, but it's all the same Linux kernel, I assume. What kind of problems are you facing exactly?
A cursory search with my search engine of choice yields several reports of people runnung Ubuntu on bhyve on Ryzen without apparent problems, for example see this forum post at iXsystems.


----------



## kpedersen (Jul 31, 2019)

stratacast1 said:


> This could be a good option
> 
> 
> 
> ...



I use the Z420 at work and my colleague uses the Z440. They work well with FreeBSD and are extremely boring (which is certainly what you want in a server).

Interestingly we could never get the inbuilt card readers and front facing USB ports to work on Windows but they work great on FreeBSD.
If you ditch the NVIDIA shite and go with an AMD GPU, it even works well with OpenBSD/X11.


----------



## steveoc64 (Jul 31, 2019)

Solved - FreeBSD 12-RELEASE + Ryzen + Bhyve == any luck ?
					

New build here -   Older generation Ryzen2600 (6c/12t . 65W, very cheap) Asrock B450-Pro4-F (very cheap) Nvidia GT710 (very cheap) 64GB of cheap DDR4-2666 NVME drive (very cheap)   FreeBSD 12.0  running great, fantastic little headless server box for general duties and developing jailed apps ...




					forums.freebsd.org
				



^ got a thread there that details the issue - and one other member so far with the same issue.  
I have put zero effort into trying to debug it so far - will get more time on the weekend hopefully.



olli@ said:


> No, I didn't do anything special, as far as I can remember. I installed Suse, though, not Ubuntu, but it's all the same Linux kernel, I assume. What kind of problems are you facing exactly?
> A cursory search with my search engine of choice yields several reports of people runnung Ubuntu on bhyve on Ryzen without apparent problems, for example see this forum post at iXsystems.


----------



## stratacast1 (Aug 1, 2019)

kpedersen said:


> I use the Z420 at work and my colleague uses the Z440. They work well with FreeBSD and are extremely boring (which is certainly what you want in a server).
> 
> Interestingly we could never get the inbuilt card readers and front facing USB ports to work on Windows but they work great on FreeBSD.
> If you ditch the NVIDIA shite and go with an AMD GPU, it even works well with OpenBSD/X11.



I was eyeballing this guy, but apparently they have a proprietary power connector...since you have the box, if you were willing to take a moment would you confirm?


----------



## Sevendogsbsd (Aug 1, 2019)

Phishfry said:


> Let me give the skinny on HP Xeon boxes.
> Motherboard uses custom 18 pin power supply header and is non-standard gear.
> I think you can do better.
> 
> ...



Yep, poudriere server I have is an HP z800 and nothing in it is standard. Been running great when I use it but I only power it up a couple times a month . I am.not looking forward to if/when the proprietary 1100 watt PSU dies...


----------



## Sevendogsbsd (Aug 1, 2019)

This thread has made me wonder if anyone can use my spare 20gb of DDR3 ECC ram? 2gb sticks x 10. Free and I will ship to you if you are in the US.


----------



## Phishfry (Aug 1, 2019)

stratacast1 said:


> if you were willing to take a moment would you confirm?


Here is the adapter you can use for a regular power supply.








						ATX PSU 24Pin + Dual Molex IDE 4Pin to 18+10pin Power Supply Cable for HP Z800  | eBay
					

Find many great new & used options and get the best deals for ATX PSU 24Pin + Dual Molex IDE 4Pin to 18+10pin Power Supply Cable for HP Z800 at the best online prices at eBay! Free shipping for many products!



					www.ebay.com


----------



## kpedersen (Aug 1, 2019)

stratacast1 said:


> I was eyeballing this guy, but apparently they have a proprietary power connector...since you have the box, if you were willing to take a moment would you confirm?



Neither of them use anything particularly non-standard for the power. Just a standard UK "kettle" lead for these ones.

https://uk.rs-online.com/web/p/power-cable-assemblies/2621126/

Is this the power connector you meant or internally to the motherboard? (I can take some photos if you would like)

On thing to note is that if I recall (changed the GPU a long time ago) the NVIDIA card that came with it didn't come with HDMI. Only DisplayPort which is more "open" and less patent encumbered but practically a bit annoying because most monitors at work are HDMI haha.


----------



## stratacast1 (Nov 18, 2019)

I want to resurrect my thread if you guys don't mind  I have some extra money coming in and want to revisit this with the information that was shared. I'm a CPU/mobo away from a system still. I've looked at Phisfry's suggestions around the Intel E5 26XXv3/4 and like those ideas. I'm wondering though if it's possible to avoid some of the meltdown stuff. I'm seeing Intel performance dip on Linux these days too as their newer kernels come out and I wonder if that is also an overall trend to expect on FreeBSD as I think a lot of the performance hits are for security controls. If that's unavoidable, I like that option. EPYC is still pretty dang expensive, even 1st gen looks steep since it's still new. I know I'd also encounter those bhyve issues and I don't want that. What's the deal with Cavium ThunderX these days? It looks like a solid platform to invest in but I presume the cost is still too high and a 32 core machine would be way too overkill. I do want something that'll go another 5 years. Is it possible to just get a motherboard/CPU or are you forced into sacrificing a kidney on a new machine?


----------



## Phishfry (Nov 19, 2019)

stratacast1 said:


> I'm wondering though if it's possible to avoid some of the meltdown stuff.


Yes with the same performance issues.


----------



## stratacast1 (Nov 22, 2019)

I've concluded that for EPYC/ARM there is nothing for me. Maybe one day the market will change! But it's not today. So I purchased an HP Z440 like I was considering a long while back. Perfect timing too. Xeon 1650v3 and 64GB of ECC memory for a solid price. I figure I can just drag and drop my drives in the new system and it should work properly (I hope)


----------



## Sevendogsbsd (Nov 22, 2019)

Wow, I have a z800 I am trying to unload: Xeon x5550 (?) (2 x 6 core) and 96GB ram. Not sure where you are on the planet though, lol.


----------



## stratacast1 (Nov 22, 2019)

Sevendogsbsd said:


> Wow, I have a z800 I am trying to unload: Xeon x5550 (?) (2 x 6 core) and 96GB ram. Not sure where you are on the planet though, lol.



Haha it's all good  I liked the z440 because the CPU in it doesn't have too high of a TDP. I can also get an L model in there too if I want... I wanted something newer for the faster RAM speeds, PCIe3.0, SATA III, and I'm pretty sure the CPU supports h264 encode/decode. If not, I have an nvidia card that has that + h265


----------



## arcadian (Nov 22, 2019)

There's some market segmentation confusion going on on the Intel side for that generation.  I have two unusual system setups, one is an Xeon e3-1240 V5 and the other is a i7-6850K.  You tell me which one looks like a server to you.
The first one, the Xeon, has a C236 motherboard (MSI Workstation) which supports ECC memory and Xeon V5 chips, however in quality it looks almost identical to other MSI Z170 boards that I've seen (which in fairness are really nice boards) with the typical four RAM slots and one LAN slot and six SATA ports and an m.2 slot.  In other words it looks more like a gaming board than a server board.  And the Xeon itself is a 4/8 CPU, basically an i7-6700 with a little more juice.  It'll fit in a normal 1151 slot but won't boot up (needs the C232 or C236 setup).

The second setup, with the i7, uses a Asus X99 Deluxe motherboard which looks like a real server board - it has 8 RAM slots and 2 LAN ports and 8 SATA ports.  It doesn't come with an m.2 slot but included a PCIe adapter card.  The (giant) LGA 2011-3 CPU is 6/12, which was high end for the time and still on par with i7-8700s and first gen Ryzen 7s.  However the board doesn't support ECC RAM unfortunately.


----------

