Histologic grade is one of the most important microscopic features used to predict the prognosis of invasive breast cancer and may serve as a marker for studying cancer driving genomic abnormalities in vivo. We analyzed whole genome sequencing data from 680 cases of TCGA invasive ductal carcinomas of the breast and correlated them to corresponding pathology information. Ten genetic abnormalities were found to be statistically associated with histologic grade, including three most prevalent cancer driver events, TP53 and PIK3CA mutations and MYC amplification. A distinct genetic interaction among these genomic abnormalities was revealed as measured by the histologic grading score. While TP53 mutation and MYC amplification were synergistic in promoting tumor progression, PIK3CA mutation was found to have alleviated the oncogenic effect of either the TP53 mutation or MYC amplification, and was associated with a significant reduction in mitotic activity in TP53 mutated and/or MYC amplified breast cancer. Furthermore, we discovered that different types of genetic abnormalities (mutation versus amplification) within the same cancer driver gene (PIK3CA or GATA3) were associated with opposite histologic changes in invasive breast cancer. In conclusion, our study suggests that histologic grade may serve as a biomarker to define cancer driving genetic events in vivo.