Anyone who is capable of understanding race can be racist. That doesnt mean you are a Racist out to burn crosses, you simply possess and believe racist ideas. Yes these ideas profoundly affect you and the people around you, but no they are not and should not be criminal.
There is an important and legal distinction between speech/thought and Conduct/acts.
"I never owned slaves.."
Consider the following analogy. You are a horny teenaged virgin sexist pervert. You sit in a room alone with an attractive teenage girl and your perverted thoughts. You never touch her. Have you committed a crime? No. Did you ever do anything which approached a crime? No. Should she trust you? No. If you were her father, would this lack of criminal behavior give you any peace of mind? Hell no. Every action you could possibly consider with that girl is greatly colored by the content of your thoughts - which freely belong to you. Until you can rid yourself of these untoward and immoral thoughts, it is very likely that any interaction between you and the girl will be disrespectful at best, and deplorably criminal at worst.Praxis
In order to gain trust, you must demonstrate your trustworthiness. No amount of talk about the relative purity of your thoughts is significant. And that holds true whether or not you are a sexist pervert. So, with regard to racist offenses and the presumptions racial minorities make about Anglos, the content of your character is often beside the point. That is the difficulty many whitefolks find in getting trust; they have no experience in demonstrating trustworthiness across racial lines. Nevertheless, that is a burden they must face and overcome simply because they grew up in America.Likewise, unilateral distrust of Anglos betrays the racialist presumption that they are incapable of being anything but racist 'because they are white'. This is a formula for disaster - a self-fulfilling prophesy which engenders righteous indignation from whitefolks who have done their due dilligence. It makes a broad racial reduction of whitefolks - ironically, the pot calling the kettle black. It's simply racist to assume that whitefolks are evil in this regard and that mutual trust is impossible. It destroys the fundamental premise of the Race Man, that the actions speak louder than words. Racial minorities, because they are born in America must meet the challenge of accepting good intent, but everyone must equally rise to the challenge of anti-racist action, otherwise it's just an interminable (and eventually unbearable) holier than thou war of words.
Therefore it is not sufficient to simply claim immunity from racist influence. Whether or not that is actually true, some political and/or cultural distinction must be made which is clearly anti-racist. Everyone has a unique burden of clearing demons from their minds based on the racial identities we are given in America. This is the first step in achieving - not colorblindness - but a concrete recognition of the difficulties we have stepping out of racial roles into a better American identity.