There are plenty of buses and bus routes within Taoyuan, as well as long distance buses to Taipei, Taichung, etc. There’s even a free bus system within the main urban area of Taoyuan District (formerly Taoyuan City). However, the bus system is not English friendly compared to Taipei. If you can’t read Chinese, or don’t have the patience to figure out what’s what using Google Translate, then you’d probably say screw it.
The bus website here (http://www.tybus.com.tw/default.aspx?page=BusTime) lists over 100 routes with maps, although the times and frequencies vary, of course.
For example, below is the route map for a bus that runs from the train station to the Art Center Park. You’d need to be able to read the street/stop names (Art Center = 中正藝文特區) plus figure out that the “Train Station Stop” is actually two blocks away from the station on Zhonghua Rd (中華路). Some of the buses have Chinese and English stop announcements inside, but not all.
Since Taoyuan became a special administrative city whatchamallit, it has been updating bus stops with electronic signs that display real time info for when the next bus is coming, just like in Taipei. I’m excited about that.
Although the new airport MRT currently makes very few stops in Taoyuan, there are plans for more convenient stops, but who knows how long that will take.
So yeah, if your Chinese skills are not up to the task then having a car or scooter is the most convenient way to get around town. There are always taxis in a pinch. Getting to/from Taipei by bus or train, especially if you’re located near the train station, is easy. If the OP needs help with this, feel free to ask me.