I tried a few things. It seemed to get a little bit better once I added duplicate power lines, but it was still floating around 4v - 4.2v. I checked and I’m “only” using 80 mA of power.
I tried every USB adapter I had around to see if it was a crappy adaptor.
I’m not sure if it’s a crappy breadboard or just all breadboards are like that.
I could have probably soldered up a point-to-point board, except I didn’t have the sockets for all of the part sizes, nor did I have a sufficently large board… and that whole process is annoying.
So, this is where the hack-day ended. But Hack Day is there till the end of the weekend, really.
Eventually, I asked on the adafruit discord and madbodger suggested and it was suggested to go to board-level regulation. So I wired up a 12v power supply, so the power switch switches 12 volts, and then put a pair of 7805 regulators, one for each board. And that got me to the point where I had happy 5v rails.