I’ve never really had any issues with my merino base layer and that’s all I wear in cold weather. It could be your jersey or jacket isn’t breathable?
Typically, base layer is there to help regulate your body temp or whick sweat, however, if your outer layer isn’t really windproof, you are in for quite a cold descent.
For point of reference, on Xmas day, I had a merino base layer, long sleeve fleece layer and a thick wind breaker jacket and I was still freezing going down the hill.
One item I always bring is a helmet cover. While going up I store it somewhere.
But if it is raining, if it is getting dark, or when going downhill i put it always on, sometimes even in the summer.
Like this:
https://www.amazon.de/VAUDE-Helmet-Raincover-Wasserdichter-Helmüberzug/dp/B00ONW13ZC/ref=sr_1_4?keywords=Helmet%2BCover&qid=1703626284&s=sports&sr=1-4&th=1
Pro Tip for European winters:
Some winters ago I always wore my skiing helmet when riding to work during sub zero temps 
Just wanted to jump in here and say I have no problem myself with my own Merino layers. Are yours 100% or some kind of blend? I buy mine from a UK company called Vulpine…although they don’t apparently, currently, ship here (I asked). But if you can get them, they’re great. They wick like crazy. I’m ridiculously comfortable in them, whatever the weather.
For very cold temperature can try these under long finger gloves. Maybe there is also something similar for feet. Some plastic bag or something. Can take off easily if get too warm.
Great setup! Layering is key, especially with merino and windproof gear. For those colder climbs above 800m, winter gloves are a must. How did the setup work for you on your ride?
Typically works fine for me. The most I ever wear is a long sleeve merino base layer, windproof + fleece lined jacket. regular bibs, double layered fleece leg warmers, merino socks and shoe covers.
The winter we’re having is a bit crazy, but I still haven’t picked up a pair of gloves. Just not used to descending with gloves.