I’ve said it before–Taoyuan is not all that bad. Immediate downtown–not so good; areas like Da Yo Road are somewhat better. I managed 11 years living from Ping Jen to Nankan, and enjoyed my time. Tiger Head Mountain stretches quite a ways for hiking, and you can head over to any of the universities (Nei Li/Chung Li) for sports such as basketball, soccer or even ball hockey. I found it much easier to practice Chinese on a daily basis as well. Taipei is not that far away for museum/cultural trips–wasn’t a place I ever wanted to live.
Keep an open perspective–I know quite a few others who have lived in the area for 15+ years and enjoy themselves.