I am fond of local tour groups. I am lazy and I do not drive, so take me, lodge me in a 5 star hotel, feed me 3 meals a day plus snacks, I will be happy as an oyster in its shell and give you a generous tip.
That said, I do speak Chinese and the experience has been mamahuhu. Most good with occassional pitfalls. But you cannot argue with the price. You can argue with the shopping stops, though. Not when they give you the whole tour in Taiwanese.
I have also taken the international, English spoken tours. Never with Mainlanders. But all tours have included the shopping stops.
That said, another option, for instance, is what we did when we went to Hualien: you can book a local guide to take you around, like a private driver. Ours was with the mingtzu/B&B where we stayed. Taidong was also very easy to move around. The B&B had bikes and maps. Most destinations you can get to by train or taxiu, and the taxi drivers will pick you up say after your swim and take you somewhere local. Definetively recommend Taidong ... take the plane, though. Book B&B on the Internet and get airport pick up.
Look for a B&B around Wuling farm with pick up. Maybe Kaohsiung or Tainan, Southern destinations untouched by Mainlanders. Take the HSR, then taxi/get picked up by hotel/hostel. I want to stay in that Batman themed hotel down South.