Why are affirmative action programs legal and constitutional?

If they were applied to situations where the White populace becomes the minority as in some places of California and PR. then it would be valid but it is only applied to blacks and latinos! Therefore aa is not legal or constitutional!