> Companies like Rosewill sell ATX cases that can scratch that itch.

Have you looked at what they cost? Those cases alone cost as much as a used server. Which comes with a case.

> What about PCI Express card? Regular ATX computers are expandable.

As mentioned higher up, they run out of lane count in a hurry. Especially when you're using things like used Connect-X cards

A rackmount case from Rosewill costs a couple of hundred bucks or so, new. And they'll remain useful for as long as things like ATX boards and 3.5" hard drives are useful.

I mean: An ATX case can be paid for once, and then be used for decades. (I'm writing this using a modern desktop computer with an ATX case that I bought in 2008.)

PCI Express lanes can be multiplied. There should frankly be more of this going on than there is, but it's still a thing that can be done.

Consumer boards built on the AMD X670E chipset, for instance, have some switching magic built in. There's enough direct CPU-connected lanes for an x16 GPU and a couple of x4 NVMe drives, and the NIC(s) and/or HBA(s) can go downstream of the chipset.

(Yeah, sure: It's limited to an aggregate 64 Gbps at the tail end, but that's not a problem for the things I do at home where my sights are set on 10Gbps networking and an HBA with a bunch of spinny disks. Your needs may differ.)