Experiencing racism apparently fatigues that hunk of meat in your head.

The problem is that we have limited cognitive resources, so when we are solving one problem, we have difficulty focusing on another at the same time. Some psychologists reason from this that subtle racism might actually be more, not less, damaging than the plain antipathy of yesterday, sapping more mental energy. Old-fashioned racism — a “No Negroes Allowed” sign, for example — is hateful and hurtful, but it’s not vague or confusing. It doesn’t require much cognitive work to get it. But if you’re the most qualified candidate for a job, and know it, and still don’t get the job for some undisclosed reason — that demands some processing.
[...]
It did, at least for blacks, and more than the overt racism did. As reported in the September issue of Psychological Science, black volunteers who had witnessed unfair but ambiguous hiring decisions did much less well on the Stroop test, suggesting that they were using all their mental resources to make sense of the unfairness. Interestingly, white volunteers were more impaired by overt racism than by the more ambiguous discrimination. Salvatore and Shelton figure this is because whites rarely experience any racism; they do not even notice the subtle forms of racism, and are thrown off balance when they are hit over the head by overt acts. Many blacks, by contrast, have developed coping strategies for the most hateful kinds of racism; it’s the constant, vague, just-below-the-surface acts of racism that impair performance, day in and day out.