I agree with Geoff Pullum at Language Log that the phrase “person of color” is objectionable. But it does have one useful function: it highlights markedness relations. After all, white people have “colour” too – pink, for example. (Steve Biko‘s jokes about apartheid and the designation “colored” are a good source here.) The only way people could have come up with a phrase “person of color” is by highlighting the marked relationship between minority and majority skin colour. It’s a good reminder that the Sapir-Whorf hypothesis is bunk. Changing the term doesn’t get rid of discrimination.
Perhaps we should try to make this formulation a bit more widespread. If we’re stuck with the phrase “person of color”, how about adding to it person of gender? A computer geek might be a person of RAM? Any more?