This part can really be a challenge though, because some hospitals refuse to give an English birth certificate without the HHRO first issuing the English household registration document, and one of the only ways to get that document with the “real” English name on is to first get the hospital certificate with it on. You can see why it can turn into a dumb loop.
I think your chances of getting the hospital to issue whatever name you want increases if you go to a small clinic. My wife gave birth at 曜生 in Xinzhuang and they gave it without issue. They also have clinics all over Taiwan I believe. Unfortunately giving birth in a clinic rather than a large hospital can result in far from favorable experiences, as I documented here: