Batteries are easy to get thanks to Cameron Sino! They make brand new batteries for old devices from cameras to PDAs.
As for parts, depends! Some you can easily find broken/untested units for cheap to grab parts off of. Others like my daily driver, the Sony Clié PEG-UX50(It even flips around to hide the keyboard!), is rarer since it was top of the top of the line, and therefore wicked expensive ($700! Today that’s worth $1,200!) Working units can be found but command high prices, and it needs a special dingus cradle for charging. Without the cradle, you can’t charge as it doesn’t take USB power.
A lot of the time, untested units usually just need a new battery and they kick right back on. Every untested device I’ve bought was much cheaper than working, and one fresh battery later it starts right up.