Neurofibromatosis type 1 (NF1) is one of the most common human genetic diseases affecting the nervous system and predisposes individuals to cancer, including peripheral nerve sheath tumors (PNSTs) and astrocytomas. Modifiers in the genetic background affect the severity of the disease and we have previously mapped two modifier loci, Nstr1 and Nstr2, that influence resistance to PNSTs in the Nf1−/+;Trp53−/+cis mouse model of NF1. We report here the analysis of Nstr1 in isolation from other epistatic loci using a chromosome substitution strain, and further show that a modifier locus (or loci) on chromosome 19 influences resistance to both PNSTs and astrocytomas. This modifier locus interacts with sex, resulting in sex-specific modification of tumors. Allele variability on chromosome 19 affects both the timing and the penetrance of the growth of different tumor types associated with NF1, specifically PNSTs and astrocytoma. These results indicate that modifiers of cancer susceptibility interact and affect tumorigenesis under different genetic conditions and demonstrate the power of chromosome substitution strains to study genetic modifiers.