In most cases you stay in Taiwan and renounce your citizenship here.
Cooling tower ( ex USA ) Gov Attache ( ex UK ) and myself ( Ex Aus ) were all able to do so.
Poagao’s situation was also unusual and a lot way back when too.
SO you need to ask your country of citizenship about the renunciation procedure.
It does take time to do so do plan on running around the word for awhile. 