I was in a similar mindset first round here after four years and in my late 20s. I knew my prospects back home would be dimmer the longer I stayed and had gotten what I needed out of learning Chinese. Languages look good on your resume and can be a great addition to your main skills. But for most, they’re not a career by themselves unless you want to translate/edit etc.
I applied for grad schools back home with the goal that I would eventually be able to use language skills I had picked up in my future career and maybe come back to Taiwan one day. Got into one of my target schools then focused on an industry that I wanted to be in and used a friend’s connection to get an unpaid internship at a big company during the first year. This is how I got my foot in the door.
I then followed up with the connections I made at the company until graduation and got a job offer. Worked harder than everyone around me and quickly moved up. Along the way, received offers to relocate to Shanghai and HK. Turned them down for personal reasons at the time. (Glad I did ). Eventually I found a niche within my industry where Taiwan was a key part and convinced my company to move my team and build an office here.
Second time around in Taiwan life is much better. The more established you are in your career the more opportunities seem to find you. I’ve had companies/recruiters reach out to me and consider my experience here an asset. My network here is now better than ever with friends from the first round, classmates, former co-workers, and people in the industry. Enough to where if I wanted to go out on my own, I could.
In order to answer the question of what is the right time to leave, you need to identify what your goals are. My suggestion would be to not just pick a career, but pick an industry and start building your connections right away. People loving sharing their experience and the more you reach out the more opportunities find you.
Good luck.