A lot of people start on used small form factor desktops like the ASUS PS52. Other common ones are the Lenovo M series tiny desktops, Dell Optiplex micro desktops or Intel NUCs. These can sometimes be found used for sub $100 a piece from businesses updating their fleets. They can struggle a bit doing stuff that needs some CPU grunt, like live plex transcoding, but are decent otherwise, especially considering the low power draw. You might want to consider spreading that software demand over two or three of them.
I started one on Linode, a 2gb shared vps. Seems to be running fine so far. Cost per month is $12.
My understanding is that as the amount and speed of memory increases, the usefulness of ECC in detecting and preventing the types of errors that can cause a crash or corrupt a file goes up.
But for home use it’s probably more useful to focus on storage redundancy and backups, or a UPS to keep things running during power blips/outages.
Tons of good options in the used enterprise market. 3-5 years old, usually some paths for basic upgrades, as well as a flood of part availability from all the other similar systems being off boarded that were broken and not resellable. Laptops can be a bit roughed up, but full sized and sff desktops are usually in great condition.