I think it's particularly difficult for people who are told by the people around them that they aren't very good. When a confident person is confronted by a hard maths question, her thought processes go something like 'OK, Don't panic, you've been through this before. Time to concentrate. Now what worked last time...' Someone with exactly the same level of maths ability who thought 'This is impossible, I'm no good at maths, hurry on to the next question' would get a lower mark, despite being just as clever.
Here's an interesting study which seems to support my theory:
Dr. Friedman and his colleagues compiled a brief test, drawing 20 questions from the verbal sections of the Graduate Record Exam, and administering it four times to about 120 white and black test-takers during last year’s presidential campaign... On the initial test last summer, whites on average correctly answered about 12 of 20 questions, compared with about 8.5 correct answers for blacks. But on the tests administered immediately after Mr. Obama’s nomination acceptance speech... black performance improved, rendering the white-black gap “statistically nonsignificant,” he said.
I think the same goes for women, the same goes for working class people. If you can make people more confident, if you have a theory that differences between people are flexible, then people can do much better. Which actually proves the point.
ETA - obviously that study is only a few days old and has not been peer-reviewed yet. I'm not claiming it literally 'proves' anything. I just mean this is the sort of thing I was talking about.