On The Exclusion Of Trans And Intersex People From “Nationally Representative” Surveys

Dr. J. SumerauDr. J. Sumerau is an Assistant Professor of Sociology at the University of Tampa.  Zir teaching, research, and activism focuses on intersections of sexualities, gender, religion, and health in the experiences of sexual, religious, and gender minorities.  In this post, Dr. Sumerau raises the provocative question: why do we call surveys that exclude certain populations — in this case, trans and intersex people — “nationally representative”?

What Does “Nationally” – As In, Nationally Representative – Actually Mean?

A few months ago, a student asked me an interesting question for which I had no answer at the time. As I do each fall, I was teaching a course on the sociology of sexualities, and as I also do every fall, I was showing students some statistical profiles of sexual and gender minority communities, issues, and concerns so that we could discuss these patterns and they could learn how sociologists utilize statistics in the study of sexualities. After class, one of my transgender students came up to the front of the room and asked, “Why do sociologists call their surveys nationally representative? Are these surveys from nations that do not have trans people?”

Despite my own personal and professional background related to transgender politics, scholarship, and experience, I must admit that I was stumped. My first inclination was to offer the standard “graduate course in statistics” answer concerning statistical weighting, government demographic tables and sources, and measurement strategies. But, I realized right away that none of these answers would suggest that our surveys actually represent any nation of which I’ve ever heard in the concrete world (i.e., a nation where only males and females exist). As a result, I decided to forgo my first inclination, and answer honestly. I told the student (as I had been told years before) that scientists (physical and social) have historically ignored and/or demonized the existence of intersex and transgender people. Not surprisingly, the student understood this, but said, “so, they’re not actually nationally representative, right?” After I agreed, the student asked, “then why do we call them that?” I have yet to find an adequate answer other than transphobia and/or cisgender privilege.

By transphobia and/or cisgender privilege, however, I do not necessarily mean research has consciously or intentionally erased transgender and intersex populations, though this may also be the case. Rather, I am observing that we live in a society historically constructed via the elaboration of sex and gender binaries by legal, social, political, religious, and scientific power structures and elites. As a result, much of our “knowledge” and “belief” is constrained by these artificial binaries and the entirety of social relations often implicitly or explicitly serve to reinforce these notions of “what counts” and “what should be.” The ability to call a data set representative of a nation when it does not contain transgender or intersex people thus (best I can tell) emerges as a result of internalizing the promotion of these binary “knowledges” and “beliefs” throughout our social world.

Let me be clear, I have attempted to find another answer to my student’s question throughout the past few months in many different ways. First, I spent considerable time reading everything that I could find on statistical theory, survey design, and methodological practice, but in none of these sources could I find a reason that we would call something that did not represent any nation “nationally representative.” The best answers that I could find suggested that since our government erased transgender and intersex people in its data collection, we scientists just did the same in our surveys (or vice versa). Next, I began just casually asking colleagues the same questions that my student asked, and their responses fell into three categories. Some people (like me) responded by experiencing “oh shit” moments and then saying that they guessed it reflected transphobia or cisgender privilege built into science. Other people either (a) said “that’s just the way we’ve always done it” or (b) got frustrated and didn’t want to talk about it. Finally, I began casually asking this question at conferences (three of them so far), but once again I got the same two answers (probably transphobia or religious impressions of “always been this way”) coupled with more angry and frustrated reactions wherein people didn’t want to talk about it or simply dodged the question.

I must admit I am especially fascinated by the angry, frustrated, and/or unwilling to talk about it reactions I have received because these responses are identical to the responses of preachers and other devout believers whom I encountered when I asked questions in church as a child. Although I cannot be certain, I think these reactions likely stem from (a) people’s faith in “representative statistics” or “statistical generalizability,” which leads them (like people with faith in other secular or religious forms of knowledge and prophesy) to lash out at anything that challenges their beliefs and assumptions about “what is real” and “what is right,” and/or (b) people’s realization that, by calling these surveys “representative,” we are participating in the erasure and marginalization of transgender and intersex people, which leads them (like people who benefit from other dominant social norms) to face difficult questions about their role – intentionally or otherwise – in the pain and suffering of others. In either case, I am rather amazed by just how “faithful” or “dogmatic” many people are when someone questions normative assumptions about statistics and/or surveys.

As I continue to seek answers, however, I have run into a couple of exceptional responses. In such cases, people avoided the question at first by pointing out that nationally representative surveys always leave out some groups (i.e., the homeless or smaller populations like those found in new religious movements), so it did not necessarily say anything about transgender or intersex people. Of course, I then asked them why we called something “nationally representative” if segments of the nation (i.e., whichever ones they had just noted) were left out of the sample. Why did we not simply call these “selected” or “chosen” or (as we do with some other surveys) “convenience” samples? At this point, I once again was unable to get an answer to the question (i.e., they generally became angry, frustrated and/or didn’t want to talk about it anymore), but I was able to point out that their first response demonstrated the point of the question. Whether or not the absence of trans or intersex people says anything about transphobia or cisgender privilege, there is still no empirical reason that I can find to call something that does not represent any actual nation “nationally representative.”

In fact, this practice is incredibly problematic if we seek to study the “actual” rather than some “imagined” social world. If we call something nationally representative that leaves out portions of said nation, for example, we are symbolically saying these people either (a) do not belong in our nation, (b) do not matter in our nation, and/or (c) are not worth our attention, concern, or respect as researchers. Within sociology, we have a term for such processes when done by other (i.e., not our own surveys) means: symbolic annihilation (i.e., the symbolic erasure of inconvenient or marginalized truths and communities for the sake of power and privilege). Likewise, if we call something nationally representative that leaves out portions of said nation, we are not studying the empirical world, but rather engaging in creative writing about a possible world we have created to fit our own needs. When others (i.e., not physical or social scientists) do this, we typically call it religion or spirituality instead of science. While these suggestions may be especially frustrating or anger-inducing for many of us (especially given the prominence and prestige of “nationally representative” terminology in our existing academic structures), they are likely considerations that all scientists should consider and debate if we hope to avoid becoming just another religious tradition.

Closing Thoughts

The combination of these experiences has led me to stop using the phrase “nationally representative” whenever possible until I see surveys that actually reflect empirical populations within our world. Instead, I have begun referring to these surveys (i.e., the General Social Survey, Add Health, and others) as either “Cisgender Representative Surveys” or “Biblically Representative Surveys” since they only contain cisgender populations (i.e., so the weighting might make them representative of these populations), and they do actually reflect what the Bible says our world looks like (i.e., males and females only). At other times, I simply point out that every sample that does not contain a representation of all social groups is simply a “convenience” or “self selected” sample. Not surprisingly, some people have responded with cheer at my new terms while others have become very uncomfortable or angry. In both cases, however, I am attempting to create space where we may begin to wrestle with the question my student asked me last fall.

As a result, I close this post by asking all of you the same question: why do we call these surveys “nationally representative,” and what does that say about the place of transgender and intersex people (as well as other groups that are marginalized, small, or for some other reason not part of the “nation” represented despite their existence within empirical nations we attempt to “represent”) both within society and within science? Further, I call upon readers to think about resources that could be developed to help scholars who seek more “representative” survey instruments. How could we go about constructing surveys in ways that encourage transgender, intersex, and other underrepresented populations to participate? Obviously, one way would be to stop calling surveys that leave out populations “representative.” But, beyond this linguistic shift, what concrete ways could we go about actively seeking to include all possible elements of populations (national or otherwise) in our survey designs?

4 thoughts on “On The Exclusion Of Trans And Intersex People From “Nationally Representative” Surveys

  1. Hmmm…the problem with the exclusion of trans/intersex people is, actually, different from the problem with the exclusion of homeless people, and I would argue that it is less true that trans/intersex PEOPLE are excluded from nationally-representative surveys than that accurate gender information is not collected.

    When survey organizations construct their sampling frames, they do so based on phone numbers or addresses–things that homeless people often do not have, but which (non-homeless) trans/intersex people DO have. Thus, trans/intersex people are indeed contacted by survey-takers, rather than being initially excluded.

    I would not be surprised to learn that trans/intersex people are less likely to agree to participate in the survey once contacted, but this does not mean that the surveys are convenience samples–rather, it means that the surveys suffer from non-response bias.

    In some cases, respondents can choose to participate without providing a gender identity–in the 2008 General Social Survey (just what I happened to have on hand), information on the respondent’s sex is missing for 40 participants, 1.9% of the sample. This situation deprives researchers of accurate data, of course, but does not quite mean that trans/intersex people are excluded from the sample. Undoubtedly, many trans people who have transitioned into a new gender identity provide that new gender identity to surveyors.

    A bigger problem, at least in the case of the GSS, is that interviewers may code the “sex” variable without even asking the respondent (see GSS questionnaires at http://www3.norc.org/projects/General+Social+Survey.htm/). But again, this does not mean that trans/intersex people are excluded from a nationally representative survey–it means instead that while their answers to other sorts of questions, like those on public opinion, can be used, researchers should assume that the “sex” data is particularly erroneous and of course also that these surveys cannot be used to provide information about the trans/intersex population. These latter issues could easily be addressed by changing questions and adding questions to the survey, and guidelines for how to propose additions are available at http://www3.norc.org/NR/rdonlyres/1176E201-8EB0-43D3-B80A-9078602815E5/0/gssguidelines.pdf

    Like

    • I appreciate you sharing your thoughts.

      As noted in the post, however, my ultimate concern is why we would call something a representation of x (nation, group, etc) when it leaves out parts of that group – at the very least this is inaccurate language, but in the case of marginalized groups it can be dangerous. The fact that we have to wonder if there might be trans and intersex people in data sets called representative of our world echoes the invisibility of these groups in contemporary society, which facilitates their ongoing marginalization.

      I should also note that your characterization of homeless communities (correct in relation to what I’ve found studying and doing statistical practice) automatically means we are not actually using probability sampling (i.e., probability sampling rests on the assumption that anyone may potentially be selected in the population, but as you note this is not the case for many homeless communities because they don’t posses the resources we use for sampling) but rather that we are calling these things “probability sampling” and “representative” when in fact they are convenience samples (i.e., whoever conveniently possesses the resources we sample from (i.e,, phones, addresses, etc) has a probability of being sampled but not everyone does) because the probability of being sampled relies upon the possession of resources and / or official forms of identification that not everyone has access to. Again, my question comes back to why do we call things representative when they leave out portions of what we say we are representing – why not just describe our surveys accurately (i.e., this is a convenience sample overall, which uses probability sampling in the case of people who possess addresses and phone numbers, but does not represent the entire population of people and does not have a probability of sampling anyone without an address or phone number) instead of pretending these data sets represent more than they actually do?

      In any case, again I appreciate the comments and thoughts and that you shared resources that I and others considering these questions regularly engage with as I hope it will be helpful in the conversation. As suggested in the post, my ultimate goal (as has been common throughout scientific her-his-our-story) is to keep working towards always making our methods and data more accurate and useful to the population as a whole.

      Like

  2. Pingback: Write Where It Hurts

  3. Good questions that have me thinking! It seems that you’ve correctly concluded that language like “nationally representative” is at best a shorthand that is easily misunderstood and misused even by survey experts. In fairness, it would be cumbersome to always phrase it as “nationally representative within the limits of my survey instrument(s), topical interests, and available resources.”

    From a different point of view, if we pursue your line of thinking further then we’ll never be collecting all of the information that we can possibly collect, we’ll always be excluding a nearly infinite number of groups or characteristics, and we’ll be left in a position where we must throw up our hands in despair and quit administering surveys altogether. We always have to make choices in what data to collect. What I think we can constructively learn from all of this is that we should be choosing to collect or omit these data on conscious and reasonable grounds that are ethically and intellectually defensible. In other words, survey researchers should have a reasonable answer to the question “Why didn’t you ask respondents if they are cis/transgender/intersex/etc. or allow them those response options?” To be clear, I think that it *may* be completely reasonable for researchers to respond with answers like “Those characteristics are unrelated to the phenomenon in question” or “We simply didn’t have the resources to adequately explore the implications for (proportionally) small groups of people.”

    (And I sincerely hope that you’ve taken some poetic license in describing how people have reacted to your questions. I don’t see anything wrong with a frank “Oh wow, you’re right – we’ve never really considered those populations and made deliberate decisions on how to collect information from and about them” kind of response but to dismiss the issue out of hand seems to be ethnically and professionally irresponsible!)

    Like

Comments are closed.