I almost always wear wool underlayers. When it cools off a bit add cotton. When it cools off alot add fleece, then windbreak outer. If it gets really cold silk underlayer, wool, fleece, down/Primaloft Gold, windstop.
Midlayers really only need to add loft, outer layers should be...