The regulation is for everyone, not just foreigners.
At least the Taiwanese teachers we had, said they were not allowed to do it officially.
But any public kindergarten my kids were in so far ignored that and did it hush hush anyway. Taiwanese teachers also know the risk and it is obviously not done in front of the security cameras.
My youngest is in his last year of kindy and he even has an English book with occasional homework learning a few words.
There seems a large opposition to those rules by parents and staff. Like why a 5 y/o shouldn’t learn how to do basic math.
Maybe some places try to hire a foreigner to reduce the risk for staff. In case they get busted the others can wash their hands in innocence having a scapegoat.