It depends. Sometimes people do, sometimes they don't- if there are layers, it's generally only one underneath. I would imagine depending on how hot it was outside, really.

Heh. Well. Mostly it's your skinniness that is to be damned, actually, and I feel your pain on that one. Wearing the upper half pulled up a little rather than clinging would help immensely- this is how most Japanese men wear it anyway.