Publications

Zhang, Simone, Janet Xu, and AJ Alvero. 2025. “Generative AI Meets Open-Ended Survey Responses: Research Participant Use of AI and Homogenization.” Sociological Methods and Research 54, no.3 [Code and Data]

The growing popularity of generative artificial intelligence (AI) tools presents new challenges for data quality in online surveys and experiments. This study examines participants’ use of large language models to answer open-ended survey questions and describes empirical tendencies in human versus large language model (LLM)-generated text responses. In an original survey of research participants recruited from a popular online platform for sourcing social science research subjects, 34 percent reported using LLMs to help them answer open-ended survey questions. Simulations comparing human-written responses from three pre-ChatGPT studies with LLM-generated text reveal that LLM responses are more homogeneous and positive, particularly when they describe social groups in sensitive questions. These homogenization patterns may mask important underlying social variation in attitudes and beliefs among human subjects, raising concerns about data validity. Our findings shed light on the scope and potential consequences of participants’ LLM use in online research.

Abascal, Maria*, Janet Xu*, and Delia Baldassarri. 2021. “People use both heterogeneity and minority representation to evaluate diversity.”Science Advances 7: eabf2507 [Code and Data]

*equal authorship, listed alphabetically

The term “diversity,” although widely used, can mean different things. Diversity can refer to heterogeneity, i.e., the distribution of people across groups, or to the representation of specific minority groups. We use a conjoint experiment with a race-balanced, national sample to uncover which properties, heterogeneity or minority representation, Americans use to evaluate the extent of racial diversity in a neighborhood and whether this assessment varies by participants’ race. We show that perceived diversity is strongly associated with heterogeneity. This association is stronger for Whites than for Blacks, Latinos, or Asians. In addition, Blacks, Latinos, and Asians view neighborhoods where their own group is largest as more diverse. Whites vary in their tendency to associate diversity with representation, and Whites who report conservative stances on diversity-related policy issues view predominately White neighborhoods as more diverse than predominately Black neighborhoods. People can agree that diversity is desirable while disagreeing on what makes a community diverse.

Xu, Janet, Aliya Saperstein, Ann Morning, and Sarah Iverson. 2021. “Gender, Generation, and Multiracial Identification in the United States.” Demography 58, no.5: 1603-1630 [Code and Data]

Multiracial self-identification is frequently portrayed as a disproportionately female tendency, but previous research has not probed the conditions under which this relationship might occur. Using the 2015 Pew Survey of Multiracial Adults, we offer a more comprehensive analysis that considers gender differences at two distinct stages: reporting multiple races in one's ancestry and selecting multiple races to describe oneself. We also examine self-identification patterns by the generational locus of multiracial ancestry. We find that females are more likely to be aware of multiracial ancestry overall, but only first-generation females are more likely than their male counterparts to self-identify as multiracial. Finally, we explore the role of racial ancestry combination, finding that multiracial awareness and self-identification are likely gendered differently for different segments of the mixed-race population. This offers a more nuanced picture of how gender interacts with other social processes to shape racial identification in the United States.

Iverson, Sarah, Ann Morning, Aliya Saperstein, and Janet Xu. 2022. “Regimes beyond the One- Drop Rule: New Models of Multiracial Identity” Genealogy 6, no.2: 57

The racial classification of mixed-race people has often been presumed to follow hypo- or hyperdescent rules, where they were assigned to either their lower- or higher-status monoracial ancestor group. This simple framework, however, does not capture actual patterns of self-identification in contemporary societies with multiple racialized groups and numerous mixed-race combinations. Elaborating on previous concepts of multiracial classification regimes, we argue that two other theoretical models must be incorporated to describe and understand mixed-race identification today. One is “co-descent”, where multiracial individuals need not align with one single race or another, but rather be identified with or demonstrate characteristics that are a blend of their parental or ancestral races. The other is the “dominance” framework, a modern extension of the “one-drop” notion that posits that monoracial ancestries fall along a spectrum where some—the “supercessive”—are more likely to dominate mixed-race categorization, and others—the “recessive”—are likely to be dominated. Drawing on the Pew Research Center’s 2015 Survey of Multiracial Adults, we find declining evidence of hypo- and hyperdescent at work in the United States today, some support for a dominance structure that upends conventional expectations about a Black one-drop rule, and a rising regime of co-descent. In addition, we explore how regimes of mixed-race classification vary by racial ancestry combination, gender, generation of multiraciality, and the time period in which multiracial respondents or their mixed-race ancestors were born. These findings show that younger, first-generation multiracial Americans, especially those of partial Asian or Hispanic descent, have left hypo- and hyperdescent regimes behind—unlike other young people today whose mixed-race ancestry stems from further back in their family tree.