Humm that's about what I wear, I use a base layer of Duofold long jons (Outer layer: 55% polyester/45% wool, Inside Layer: 100% Polyester), a middle layer of Adidas sweatpants (the shiny slick kind) and an outside layer of Viking non-insulated ski pants. That combination works just fine for me. What could be your problem is the base layer or the middle layer. You might know this already, but, as a base you want something to wick the moister away while still holding its insulating value, polyester is good at this. If you are using that then maybe you want to thicken your middle layer with either heavier pants or a light sweat-pant (I'm thinking the kind you get from Walmart). If those options don't work you could always sew some pockets into your long jons and put some hand warmers in them
Hope this helps