Need 16x-8x configuration at same time. Are there any consumer boards?

[philb2]

Over on LinkedIn, there is a market forecast that predicts

PCIe Switches Market Size is Growing at a CAGR of 11.9% with Worth ~ US$ 1764.6 million by 2032 [Exclusive 98+ Pages Report] https://www.linkedin.com/pulse/pcie-switches-market-size-growing-cagr-119-vysfcv​

If you believe this forecast, then there is no way that Broadcom can defend its market share. Customers simply won't stand for it, and they will go out of their way to second-source or multi-source these parts. Look at the list of competitors.

But, but, but, the report only covers up to Gen3 . Hard to believe.

 

[ND40oz]

A lot of boards just come down to how certain I/O decisions are made. Do I think i'm really going to make use of USB4? So far, I haven't. Though being able to run a display off the iGPU directly from one of the Type C ports is pretty nice.

It doesn't necessarily have to be the iGPU running over the Intel thunderbolt controller that they call USB4. I run my 7900XTX through the thunderbolt controller to my thunderbolt monitor using the displayport input on the motherboard, one cord and my keyboard and mouse plug into the monitor. Works great for KVM functionality with another computer hooked up to the displayport input on the monitor.

 

[criccio]

It doesn't necessarily have to be the iGPU running over the Intel thunderbolt controller that they call USB4. I run my 7900XTX through the thunderbolt controller to my thunderbolt monitor using the displayport input on the motherboard, one cord and my keyboard and mouse plug into the monitor. Works great for KVM functionality with another computer hooked up to the displayport input on the monitor.

No, but it depends on how you're accomplishing that. The X670E Taichi doesn't have DP input to passthrough video over USB4/TB so you'd have to rely on just telling Windows to use the XTX for rendering but outputting video via the iGPU.

Some boards do have DP inputs to purely passthrough the dGPU via that output. Asus's Creator boards come to mind.

 

[Nobu]

Looks like nobody's run any 3dmark benches with a 7945wx yet. There's some 5945wx results, they get a little over half of a 7900's CPU score in timespy, with the same GPU and clocked a good bit lower. A 7945wx might get +75% CPU score? Not sure of the exaxt generational improvement, I know it's a good bit, plus the clock improvement.

Here's a couple I picked to compare, tried to match and avoid huge overclocks.

 

[ND40oz]

No, but it depends on how you're accomplishing that. The X670E Taichi doesn't have DP input to passthrough video over USB4/TB so you'd have to rely on just telling Windows to use the XTX for rendering but outputting video via the iGPU.

Some boards do have DP inputs to purely passthrough the dGPU via that output. Asus's Creator boards come to mind.

View attachment 647210

Correct, that's exactly why I bought the X670-E Creator. But it doesn't just pass the dGPU, it also gives you a full bandwidth thunderbolt connection to any thunderbolt devices you want to connect. One active thunderbolt cable connected and you have access to all the peripherals you want.

 

[philb2]

After reading this thread, I'm afraid that any hope for future HEDT boards is just pissing into the wind. Don't mean to be a downer, but that's how I feel.

 

[TheGardenTool]

It doesn't necessarily have to be the iGPU running over the Intel thunderbolt controller that they call USB4. I run my 7900XTX through the thunderbolt controller to my thunderbolt monitor using the displayport input on the motherboard, one cord and my keyboard and mouse plug into the monitor. Works great for KVM functionality with another computer hooked up to the displayport input on the monitor.

What motherboard do you use? Mine has a DisplayPort in but I haven’t messed with using it. I’m guessing the ProArt X670e?

Zarathustra[H] I did find an Atto product that claimed to use TB4 to some type of 40Gb fiber but it’s discontinued and seemed to be pretty expensive second hand still. And I didn’t find any solid confirmation that it actually runs at the full 40Gb. Lots of 25Gb options through TB3/4 though i think that would be slower than just running your card on a limited x4 slot.

 

[ND40oz]

What motherboard do you use? Mine has a DisplayPort in but I haven’t messed with using it. I’m guessing the ProArt X670e?

That's the one, I always bought HEDT boards in the past and that was about as close as I could get this time around.

 

[crazycrave]

Kind of why I jumped to AMD was to carry on with Crossfire thinking they may keep it going building GPU's also, B350 couldn't compete with x58 with the same cards and driver because of the x8 by x8 setup, times have changed with cards like the RX6600 being an x8 card if AMD made a working driver I would love to see them scale in an x8 by x8 setup.

 

[philb2]

That's the one, I always bought HEDT boards in the past and that was about as close as I could get this time around.

So after reading a bit about the ASUS Creator X670e, I wish I had gotten this board instead of the ROG Strix E-A x670e.

 

[crazycrave]

View: https://youtu.be/EDO-Fh2nxSs?si=3MvIHBKRN8z2dvZc

 

[ND40oz]

View: https://youtu.be/EDO-Fh2nxSs?si=3MvIHBKRN8z2dvZc

Yeah, unfortunately that board alone is more than a Asus Creator x670e/7950X combo and then you need to buy a Threadripper Pro.

 

[Zarathustra[H]]

View: https://youtu.be/EDO-Fh2nxSs?si=3MvIHBKRN8z2dvZc

Well, yeah, but it is a Threadripper Pro. We know we can get many PCIe lanes on workstation boards, but the problem with workstation boards is that they need workstation CPU's, and these days workstation CPU's suck for anything but workstation loads.

What I am looking for is the old concept of HEDT. An all in one no compromises build. It can be pro-sumer workstation-like AND be a consumer game machine at the same time and excel at both. Essentially a high end consumer desktop that can both attain top low threaded speeds in consumer workloads and has some light workstation-like features like many cores and large numbers of PCIe lanes. Sandly this concept appears to be dead for about 3-5 years now.

When I bought my Threadripper 3960x in 2019 (and its predecessor, my Core i7 3930k x79 build in 2011, or any x99 or x299 system) it performed equivalently to similar consumer CPU's in consumer loads, but also had the capability of adding a ton of RAM, had lots of PCIe lanes for expansion and some extra cores. It could do everything.

Now the problem is that now I have to choose one or the other. Either consumer or workstation. Or I have to build two systems (which I really don't want to do). One for each task.

The whole point of this thread is that we want 16x-8x in a consumer board. AM5/LGA1700 (or LGA1851 for next gen Intel). This is not because of cost, but because current workstation chips are MUCH slower at lightly threaded loads than consumer chips currently are. I'd happily pay extra for a motherboard that supported the type of lane layout that I need.

There are three things working against current workstation designs compared to consumer chips. Lower clocks, poor core layouts/NUMA and the fact that with DDR5 registered and unregistered RAM is no longer pin compatible, meaning you are robbed of a lot of RAM latency when you are forced to use Registered/Buffered ECC Ram in these systems.

And the result is miserable. Anything current Threadripper (pro or not) EPYC or Xeon just plain sucks for anyhting but Server/Workstation loads.

If you try to run a game on the beastly $5000 Threadripper 7980x on a $1500 motherboard you are going to be bested - performance wise - by a $200 Ryzen 5 7600 on a $150 motherboard.

Honestly, the concept of the "no compromises" HEDT system is dead unless we start getting better consumer motherboard with better slot options, which might save it just a little bit.

 

[philb2]

Well, yeah, but it is a Threadripper Pro. We know we can get many PCIe lanes on workstation boards, but the problem with workstation boards is that they need workstation CPU's, and these days workstation CPU's suck for anything but workstation loads.

That sucks royally.

Sandly this concept appears to be dead for about 3-5 years now.

Sad.Can anyone speculate as to the reasons?

Now the problem is that now I have to choose one or the other. Either consumer or workstation. Or I have to build two systems (which I really don't want to do). One for each task.

I think a lot of guys don't have workstation job needs, but just want a powerful consumer board. That is, HEDT.

The whole point of this thread is that we want 16x-8x in a consumer board.

Yeah, me among them.

Honestly, the concept of the "no compromises" HEDT system is dead unless we start getting better consumer motherboard with better slot options, which might save it just a little bit.

But we can't just "get" these boards. ASUS, MSI, etc have to start making them and then we will buy them.

 

[Dopamin3]

The whole point of this thread is that we want 16x-8x in a consumer board.

Nobody wants this except you. Probably an exaggeration but very few people desire this.

I don't think you ever responded to this, thoughts?

A PCIe 3.0 x4 slot can deliver 4GB/s bandwidth, theoretically giving you up to 32Gbps for your QSFP+ NIC. Wouldn't that suit your needs just fine? It might not reach that full theoretical speed but I'd expect to be somewhat close. Hopefully soon we'll see PCIe 4.0 x4 NICs come out since this is the same bandwidth as PCIe 3.0 x8 and that will invalidate your need for the mythical x16/x8. Or... simply eat a 2% - 3% performance loss on your GPU to run x8/x8; which is a perfectly acceptable trade-off to get very high speed networking on a consumer platform.

 

[Zarathustra[H]]

Sad.Can anyone speculate as to the reasons?

I think it is a combination of reasons.

HEDT is really a combination of three things.

1.) More cores than regular consumer chips
2.) More PCIe lanes than regular consumer chips
3.) More RAM channels than regular consumer chips
4.) ...without the performance penalties in lightly threaded loads that workstation products have.

In the past the HEDT had its own socket. Think 9xx series Bloomfield vs 8xx series Nehalem, or Sandy Bridge-E vs Sandy Bridge, or Ivy Bridge-E vs Ivy Bridge, etc. etc.

The HEDT socket had more pins allowing it to fit more PCIe lanes and more RAM channels.

Then AMD started cramming more cores into consumer AM4 sockets, and intel followed suit.

Now all of a sudden, you've already satisfied the core-heads with #1 above without needing a specialized socket. This makes it more difficult to justify the added expense of developing a specialized socket. AMD and Intel probably figured there was no need for that interim socket when they already had workstation sockets.

And this worked for a while. My Threadripper 3960x could be either a HEDT product or a Workstation product depending on how you configured it. Stick Registered ECD RAM in it and you essentially have a workstation product. Stick high performance non-ECC unbuffered RAM in it and you essentially have a HEDT platform. You can also configure the use of cores appropriately with "game mode" to avoid NUMA issues involved with temultiple CCD's

But then DDR5 launched, and Registered and Unbuffered RAM is no longer pin compatible. So if you are making a motherboard you have to choose one or the other. Now you are forced to put Registered ram in a Threadripper build, hampering performance.

And then AMD also slowly stopped optimizing "game modes" on Threadrippers, further reducing performance in more consumer workloads,

Essentially this has turned Threadrippers (both pro and non-Pro) into workstation only platforms. Xeon is mostly the same.

 

[Zarathustra[H]]

Nobody wants this except you. Probably an exaggeration but very few people desire this.

I've had others chime in in this thread agreeing with me.

I don't think you ever responded to this, thoughts?

A PCIe 3.0 x4 slot can deliver 4GB/s bandwidth, theoretically giving you up to 32Gbps for your QSFP+ NIC. Wouldn't that suit your needs just fine? It might not reach that full theoretical speed but I'd expect to be somewhat close. Hopefully soon we'll see PCIe 4.0 x4 NICs come out since this is the same bandwidth as PCIe 3.0 x8 and that will invalidate your need for the mythical x16/x8. Or... simply eat a 2% - 3% performance loss on your GPU to run x8/x8; which is a perfectly acceptable trade-off to get very high speed networking on a consumer platform.

I've tried a few times, but I have found that most enterprise NIC's behave badly when they are not given the full number of lanes they were designed for. Not quite sure why, but it doesn't just scale with raw bandwidth figures. For instance my old first gen 10gig Intel AT2 adapters were the worst. In an 8x slot they worked fine. In a 4x slot downstream worked fine, but upstream was a horrible mess of poor and intermittent performance, even though the bandwidth of the one 4x slot should in theory have been sufficient to support one port.

There is something more going on there than just adding up raw available bandwidth. I wonder if Intel hand optimized what data goes over what lane or something like that, rather than just pooling all the available PCIe bandwidth and using it.

So, I think at the very least you'd need some sort of PCIe switching to make sure the NIC sees all 8 lanes locally, even if they are later pooled and combined to just 4 lanes upstream.