I tend to stay in either one language or the other; I don’t like switching between the two, so when I’m speaking to foreigners it’s mostly in English (there are exceptions, such as a bit of chatting at Orange last Happy Hour), and when speaking to Taiwanese people it’s mostly Chinese or Taiwanese, unless their English is native-level fluent.
This might have more to do with the company I keep, though: mostly either foreigners or ordinary local people, not many “well-travelled” locals who have lived abroad. When I do meet such people, all of the going back and forth between English and Mandarin strikes me as pretentious, perhaps because I’m not used to it. It irritates me when locals throw in (mostly inappropriate and poorly pronounced) English whenever they see a foreigner, so I try to do as little of that as possible in either language. Also, I find that I tend to shift gears, mentally speaking, when going from one language to another. Many people change their entire persona when doing so, because they take on a different character and attitude, so I find it difficult to go smoothly between one and another. Then again, I’m not exactly a social butterfly, either.