A current concern in genetic epidemiology studies in admixed populations is that population stratification can lead to spurious results. The Brazilian census classifies individuals according to self-reported "color", but several studies have demonstrated that stratifying according to "color" is not a useful strategy to control for population structure, due to the dissociation between self-reported "color" and genomic ancestry. We report the results of a study in a group of Brazilian siblings in which we measured skin pigmentation using a reflectometer, and estimated genomic ancestry using 21 Ancestry Informative Markers (AIMs). Self-reported "color", according to the Brazilian census, was also available for each participant. This made it possible to evaluate the relationship between self-reported "color" and skin pigmentation, self-reported "color" and genomic ancestry, and skin pigmentation and genomic ancestry. We observed that, although there were significant differences between the three "color" groups in genomic ancestry and skin pigmentation, there was considerable dispersion within each group and substantial overlap between groups. We also saw that there was no good agreement between the "color" categories reported by each member of the sibling pair 30 out of 86 sibling pairs reported different "color", and in some cases, the sibling reporting the darker "color" category had lighter skin pigmentation. Socioeconomic status was significantly associated with self-reported "color" and genomic ancestry in this sample. This and other studies show that subjective classifications based on self-reported "color", such as the one that is used in the Brazilian census, are inadequate to describe the population structure present in recently admixed populations. Finally, we observed that one of the AIMs included in the panel (rs1426654), which is located in the known pigmentation gene SLC24A5, was strongly associated with skin pigmentation in this sample.