Surely it is a personal choice and depends what kind of bike, speed, and what type/style of riding you do, and whether stealth is an important factor.
I think the less extra weight you add to your head, the less chance of receiving serious neck injuries. Carbon fibre offers the best strength:weight ratio.