Traveller basic tips

where should I buy Taiwan Dollar. I know inside the country I will get best rates. I am planning to carry USD. Can I walk in to the bank with USD and get Taiwan Dollar without being customer of the bank? any process which I need to follow ?
I am in Taiwai for 5 days. I know there are many options available. I am looking for prepaid SIM card. I want voice. I am not very very keen about data. Any suggestion for prepaid card ? Where should I get SIM card? Does Airport shop remain open in the night also ? How fast the card can be activated ?