How about some bilingual children’s picture books? ESLITE children’s store on Tunhwa Sth Road has some very nice ones. For example, a series introducing Chinese New Year, the mouse bride, chinese zodiac, illustrations are photos of papercuts and other art techniques, stories are in English and traditional Chinese characters, hardcovers. Wish I had one here to give you the ISBN. There are also some new ones out about Taiwan like taking the train to the countryside, farming days, and so on but they are Chinese-only.
Or if the baby is very small you could get a gift like a gold charm of the zodiac sign on leather. Easy to carry, longlasting and meaningful. My son was given one as a baby gift and it is actually very nice.