Gigabyte Blew It
About three years ago my aging & rather ancient computer well and truly died. I shopped around some and settled on a few things. One was a Gigabyte motherboard, the GA-890XA-UD3 which I am using right now as I type. It has worked just fine for these last three years with no special setup beyond updating the BIOS (for which having a lowly Sempron around is a Good Thing). The only thing it really lacks is an AM3+ socket so unless I upgrade the motherboard, I can't upgrade the CPU. And thus the (mis)adventure begins...
Last year I was building up a machine for scarletcharnel and selected another Gigabyte motherboard, the GA-990FXA-UD3, which had about everything, including USB3 and an AM3+ socket and that just worked. I didn't even need to update the BIOS. This was Rev 1.1 which as far as I know is just fine. If there is a heat/throttling issue, it's hasn't shown up. That system has great ventilation and cooling.
Thus I felt fairly safe ordering another GA-990FXA-UD3 board for myself, to upgrade belgian or at least allow upgrades (new case case with more ventilation/fans, new video card, eventually a newer CPU, and maybe more RAM). Alas, that was not the case. I tried moving everything at once and things got seriously weird. Lost trackball and keyboard, unless I plugged them into USB3 ports. Lost networking. Once, in a diagnostic boot of RIP Linux, USB2 worked but USB3 didn't - and there was no indication of why. Network worked then, but only just then. I moved everything back to get a working system, then swapped out video cards and had no issue.
My next night off was spent trying again, this time with the Sempron that jmaynard had been using for something, a couple 1 GB DIMMs of DDR3 1333 (instead of four 4 GB DIMMs of DDR3 1600), the GTX 570 (which was replaced by a GTX 760  in the working belgian) and a LiveCD. And everything just worked. What the heck was going on? But I knew that the hardware could work and the board was not automatically bad. Everything also worked fine in BIOS setup screens and such.
So I figured I'd try a part by part move and see what happened. Moving RAM should be the most trivial, uneventful thing, right? WRONG. Well, right, it SHOULD be that way. It wasn't. I pulled the 2 GB of 1333 and put in the 16 GB of 1600 and the problem(s) reappeared. What the photon? Dropped to 8 GB (2 DIMMs). Problems. Swapped those two for the other two. Problems. Tried the other two sockets. Problems. Tried only 1 DIMM (4 GB). Problems. Tried slowing the timing to 1333, and even to 1066. Problems. Tried upping the RAM voltage a bit. Problems. Put everything back to AUTO and put the 1333 back in and things worked. Put the 16 GB back in the working belgian and that works.
Looking at things, it feels eerily like the bad old days of IRQ conflicts and the weird breakages that resulted. Turns out that was what was going on.
I pulled a DIMM (8 GB) from the machine that had the Sempron and try that, so I can keep my main system working until I figure things out - or return the board. Eventually I find online that there is a setting for IOMMU that is DISABLED, but switching to ENABLED makes things work - for some. Not for me. More delays and more research and I finally find someone who had the same problem that enabling IOMMU didn't fix. But he had a solution: tell the kernel "iommu=soft" at boot time. Aha! That makes everything work. USB2 works. USB3 works (the ports work, I might need to confirm USB3 speeds rather than USB2 fallback), and the network is there and working.
What is IOMMU? Input-Output Memory Management Unit. The thingie that is supposed to prevent IRQ conflict issues in this modern, enlightened Plug & Play age. Somehow, in Rev 4.0 of this board or the BIOS, Gigabyte managed to break it in a way that Linux detection can't (yet) detect automatically compensate for. And what happened, exactly? I don't know all the true low-level details, but below 3 GB of RAM, IOMMU doesn't seem to matter very much. Thus running on only 2 GB or only mucking about in BIOS screens, all was well. Above 3 GB (I tested with 4, 8, and 16... all more than 3) it's needed. But if it isn't working quite right, there are problems anyway. The kernel message is sort of "Assume IOMMU is messed up and compensate for that."
I hope that's the only issue with this board. In my research I found that it can, now, supposedly even take the new AMD factory-overclocked (and crazy hot, power hungry) FX95XX CPUs that are rated at a staggering 220 Watts instead of a "mere" 125 Watts. I have exactly zero plans to use such a thing, but I could. That would seem to indicate any power issues (Rev 3.0 has tales of woe regarding such) have been resolved. Still, the IOMMU screwiness makes me wonder if anything else is messed up. I had been at the point of considering only Gigabyte boards since I had some weirdness with an (admittedly cheap, open box) ASRock board and Jay had something a bit odd (but since forgotten, so evidently not critical) about an MSI board. Now? Now the next time I go motherboard shopping, I probably won't be gravitating to Gigabyte. Not sure what way I will go, but I really do not need this time-sink of a headache that makes me think of the bad old days of twenty years ago.
 I did have one issue, but that was a self-inflicted thing unrelated to all this.
 I saved up for good many months to be able to get that. It still was jarring to order it.
 Nobody likes the BIOS setup for this board. It well and truly sucks. It might be worth considering another make just to not have to deal with that turkey of a setup.
 Despite the stupid Windows executable file used for BIOS updates when a zipfile would be easier all the way around, for everyone - even them.
 Gad, has it been that long?