In spite of a shared genome, males and females can have dramatically different phenotypic traits. Some sex-specific traits are also polymorphic, so that different genotypes are expressed as two or more phenotypes in one sex but not the other. Such sex-limited polymorphisms offer an opportunity to reconstruct the origin of molecular mechanisms limiting trait expression to a single sex. The Drosophila montium subgroup, comprised of 76 species, harbors diverse abdominal pigmentation: some species are sexually monomorphic, others are dimorphic, and many species have female-limited color dimorphism (FLCD), where females are either light or dark but males are always the same color. Thus, sexually dimorphic and sexually monomorphic genotypes can either segregate within a single species or be fixed between closely related species, making this group an ideal model for reconstructing evolutionary transitions between sex-specific and sexually monomorphic development. We mapped FLCD in four distantly related species from the Drosophila montium subgroup and found that it mapped to the same locus, the POU domain transcription factor (pdm3), in all cases. In D. serrata, we pinpointed a small region within the first intron of pdm3 that differs in size and sequence between color morphs. The dark allele contains putative binding sites for dsx, the transcriptional effector of the Drosophila sex determination pathway, and the HOX gene Abd-B, which controls the development of posterior abdominal segments. The region spanning this complex structural variant acts as an abdominal enhancer in transgenic reporter assays, suggesting that sex-limited polymorphism is due to intraspecific segregation of cis-regulatory variation in pdm3. In D. kikkawai, D. leontia, and D. burlai, the structural polymorphism observed in D. serrata is not present, and female pigmentation is associated with different non-coding variants upstream of pdm3. These results suggest that FLCD has originated multiple times through regulatory changes in the same locus.