I’ll always remember the western lady on an episode of Banged Up Abroad (or one of those shows) who, being arrested at the airport in Bangladesh (or somewhere) for smuggling, tried to talk her way out of it like this.
Police: You’re under arrest! 
Smuggler: But my flight is about to leave! 
Police: Forget about your flight. You’re going to jail! 
Smuggler: Oh no, you don’t understand! I can’t stay in the country. My visa expires today! 
Police: Don’t be stupid, miss! You don’t need a visa to go to jail! Ha ha ha ha ha! 
Mutatis mutandis…
Nomad: Oh no, you don’t understand! I pay tax to a foreign government, so I can’t be deemed a person working in Taiwan!
Whether or not the government will take interest is not something you can always predict. They do have a financial incentive to go after small fish, namely the fines they can impose (and the image they can project to the public – we’re keeping the country safe! etc.). People in general have an incentive to report illegal foreign workers, namely the percentage of the fines they can receive (and they may derive satisfaction even without the money, and they may be competitors and so on).