You are supposed to inform NIA - National Immigration Agency- of any change of address as they may send important information - free cancer screening, for example- to your address or, Heck, it is the law and if you don’t you will be fined…heavily. You are given a reasonable amount of time to do so.
In theory, you should get an ARC for the newborn ASAP as well, he may need medical attention or a vaccine or two and you need the ARC to register such kid for NHI - National Health Insurance. We fought hard to get newborns this right, do not take it lightly. I am assuming here the kid is produced by two foreigners and hence a foreigner, with not one parent Taiwanese. Nationality on this island is via sanguis, meaning by right of blood, not by being born here on this land, so only Taiwanese can give birth to Taiwanese.
Ah, yes, the good old days of police registration. Now they dumped The Foreigner Issue into a single new agency’s lap, the NIA. To make things even more interesting, their computers are linked with the tax office. Enjoy.