Paper Menu >>
Journal Menu >>
Sociology Mind 2013. Vol.3, No.1, 7-15 Published Online January 2013 in SciRes (http://www.scirp.org/journal/sm) http://dx.doi.org/10.4236/sm.2013.31002 Copyright © 2013 SciRes. 7 The Sociological Determination: A Reflexive Look at Conducting Local Disaster Research after Hurricane Katrina Timothy J. Haney1, James R. Elliott2 1Department of Sociology and Anthropology, Mount Royal University, Calgary, Canada 2Department of Sociology, Universit y of Oregon, Eugene, USA Email: thaney@mtroyal.ca Received October 20th, 2012; revised November 24th, 2012; accepted December 13th, 2012 This paper examines the process of collecting data on New Orleanians affected by Hurricane Katrina. It does so by focusing upon the experiences of local researchers who were simultaneously conducting re- search on and within the disaster. It also documents one research team’s attempt to generate a random sample of residents from several New Orleans neighborhoods, stratified both by racial composition and level of damage. Further, it describes the challenges associated with navigating complex bureaucracies that are themselves affected by the disaster. Results demonstrate that our methods for drawing samples from six New Orleans neighborhoods yielded highly representative samples, even in heavily damaged neighborhoods where the long-term displacement required a multi-pronged strategy that involved contact by mail, telephone, and visits to local churches. The paper concludes by making recommendations for fa- cilitating future research by locally affected researchers. Keywords: Disaster Research; Hurricane Katrina; Reflexivity; Sampling Methods; Representativeness Introduction “The important thing is not to draw up in advance a plan anticipating everything, but rather to set resolutely to work” (Durkheim, 1951). When Hurricane Katrina hit the US Gulf Coast in August 2005, it stunned the nation and elicited charitable outreach from around the globe. It also rekindled sociological interest in dis- asters. This interest stemmed from the obvious social inequali- ties laid bare by the storm and by a sense that these inequalities would continue to play a key role in the region’s recovery, long after the hurricane passed and aid stopped flowing to the region. Nowhere was this sense stronger than in the city of New Or- leans, where the hurricane and subsequent levee failures trig- gered the largest and most complete urban displacement in US history. Through media broadcasts of these events, observers throughout the nation and the world became armchair sociolo- gists, bearing witness to the links between private troubles and public issues articulated in C. Wright Mills’ classic, The Socio- logical Imagination (Mills, 1959). A review of recent research reveals that this sociological imagination is making a resurgence not only in the popular mind but also in the professional study of environmental haz- ards, generating useful insights into the social underpinnings of vulnerability in everything from heat waves (Klinenberg, 2002), to hurricanes (Steinberg, 2008), floods (Erikson, 1976), conta- minants (Snider, 2004), oil spills (Picou et al., 2004) and dis- ease (Baehr, 2005). Collectively this body of research shows how social research can improve our understanding of modern disasters and, conversely, how disasters can inform under- standing of more general processes of social organization and inequality. Less evident in this literature, however, are accounts of the obstacles that social scientists, especially those living and working within a disaster region, must negotiate in order to collect and process data needed to move from imagination to empirically grounded analysis. This movement requires a socio- logical determination that we believe deserves attention, along- side the sociological imagination. The present study offers an extended view of such socio- logical determination as it intersected with the sociological imagination to document and examine social inequalities re- vealed and reproduced by New Orleans’s short- and medium- term recovery from Hurricane Katrina. Motivating this view is a conviction that in order for disaster research to reach its full potential, it can and should work intellectually and program- matically toward incorporating the perspectives and ideas of those most affected by catastrophic events, including especially locally situated researchers. Local researchers not only have requisite skills and vested i nterest in generating knowle dge about their particular event or locale, but also having experienced the event can bring progressive contextualization to theories, me- thods and research questions advanced for study. Such contri- butions, however, face a labyrinth of obstacles that prevent lo- cal researchers from creating important and timely knowledge. In this study, we offer one view of these obstacles as we en- countered them in our own collective effort to conduct research on post-Katrina recovery within disaster-stricken New Orleans. We also discuss the strengths and weaknesses of our attempts to generate a representative sample of New Orleanians from neighborhoods experiencing vastly different damage from Hur- ricane Katrina. In addition to illuminating the sociological de- termination required to conduct social research on disaster from within a disaster, we also hope to provide a go-to resource for other social scientists who find themselves similarly attempting to design a quick-response resea rch project in their own disaster- affected area, with limited institutional resources. With these broad aims in mind, the paper focuses on two central questions. First we ask how disaster research gets done by local re- T. J. HANEY, J. R. ELLIOTT searchers looking to act quickly in an affected area, relaying experience with a range of challenges, from applying for grants, to negotiating questions of objectivity and reflexivity, sampling, respondent “capture,” and, finally, analysis and writing. While disaster research is a growing area of inquiry, with a large body of empirical findings, we know comparatively little about the research process that leads to these results. Along the way, we offer lessons learned from our own experiences. These lessons include: insights for mobilizing resources in a post-disaster setting; potential avenues for locating displaced respondents; and the utility of respondent-driven sampling techniques for increasing representativeness. The second and related question we ask is how successful standard practices for random sam- pling tend to be in more- and less-devastated residential areas within a local disaster zone. By engaging these two questions and sharing our own pro- fessional experiences along the way, we hope to provide a bet- ter foundation for future disaster research undertaken by those living and working in affected regions. In this respect, what follows can perhaps best be viewed as part disaster research and part sociology of work. It outlines one team’s efforts to organ- ize and collect a particular type of data after a major disaster. The hope is that this study will add to our understanding of disasters by illuminating how one part of a distressed commu- nity (trained sociologists) set out to learn about the experiences of those around them so that such work can continue to make even stronger contributions in the future. Institutional Challenges Facing Local Researchers in Disaster S ettings Despite the ascendance of disaster research in the wake of September 11, Hurricane Katrina, and other recent catastrophic events, existing research tends to focus attention on empirical findings and pays less attention to the research process behind the methods that generated those findings. Consequently, even as the body of empirical findings on disaster continues to grow each year, we still learn little about how to conduct research on and in local disaster settings and about the institutional mecha- nisms that facilitate and constrain such efforts. Over recent years, however, a small literature outside disaster research has emerged that begins to bring these issues to the fore, encourag- ing investigators to turn the lens inward to focus on the re- searcher and associated research process. One of the first things we saw when we began to adopt this viewpoint was the power of bureaucracy to slow and shape research. Theoretically, we know from Weber (1946) and many soci- ologists since that bureaucracies, designed for efficiency and rationality, can be very inefficient and irrational by virtue of their organizational complexity and rigidity. During a disaster or catastrophe, however, when one or more levels of bureauc- racy are missing or suspended, bureaucratic structures can grind to a standstill. Haney (2007) provides the example of the fed- eral government’s disaster food stamp program and its extreme breakdown following Hurricane Katrina. Though taxed with many times the normal application volume, social services agen- cies were unwilling or unable to suspend normal processing rules and timelines in order to provide urgently needed hu- manitarian assis tan ce. Molotch (2006), however, points out that bureaucratic break- downs such as this may be only partially explained by organ- izational inflexibility. Such breakdowns are also more likely when those working within the bureaucracy fail to feel the req- uisite compassion or sense of urgency that the situation requires. In other words, bureaucracy becomes a rationale for inaction by those who do not feel compelled to act. Molotch’s argument and Haney’s example, however, both presuppose an outside bureaucracy, such as FEMA, that remains intact during disaster. The bureaucratic failures encountered by local disaster resear- chers, by contrast, also include dispersion and temporary dis- solution of local bureaucracies, most notably University Re- search Offices and Institutional Review Boards (IRBs). Thus, as local social researchers quickly set to work to develop ap- propriate research designs, gain institutional approval, and ap- ply for external funding, they soon find themselves squeezed into a liminal space created by both the presence of outside bu- reaucratic structures and breakdown of local bureaucratic struc- tures, gaining neither power nor efficiency in the process. Paradoxically, this institutional squeeze and subsequent slow- down occurs contemporaneously with the need for researchers to get into the field quickly to begin documenting the respective disaster’s immediate and long-term impact on the local com- munity. This is one reason why Barron et al. (2009) argue that, “IRBs must be prepared to act quickly when a timely opportu- nity for research presents itself.” For this reason, Knack et al. (2006) argue that “researchers looking to collect data as close to the disastrous event as possible should contact the IRB early in the development of the study to determine if expedited review is possible and to alert the IRB of the necessity of a rapid re- view.” A lacuna in the literature, however, is the question of how timely IRB approval can and should be in the context of an extreme disaster that suspends or greatly slows operations of one’s home institution, and how strict the review process should be in such uncertain times. For example, Henderson et al. (2008) report that their IRB asked that they not collect partici- pants’ social security numbers as part of their research, yet in order to provide an incentive to participants, the accounting office (still slowed by Hurricane Katrina’s devastation) main- tained that they required these numbers to proceed. The result- ing conflict slowed the respective research project considerably, threatening loss of valuable time, data and analysis. Navigating these locally compromised bureaucratic struc- tures, often in the absence of required forms, e-mail access, and contact with necessary personnel, not only slows the research process but also puts local researchers at an initial disadvantage vis-à-vis non-local counterparts operating from within uncom- promised institutional structures located elsewhere. This initial disadvantage faced by local researchers is especially important in disaster research not only because of the time-sensitive na- ture of data collection but also because as other research teams quickly enter the field, many local residents can begin to feel like lab rats, and survey fatigue sets in (Fleischman and Wood, 2002). This issue is one that all disaster researchers must con- sider, but it also means that participants may be more eager to participate in early data collection efforts, which because of the bureaucratic issues discussed above, may be more likely to be carried out by non-local researchers. Yet another contemporaneous challenge faced by local re- searchers in disaster settings is funding. Simply put, normal grant procurement cycles make acquiring external funds quick- ly very challenging. This is why the Natural Hazards Center runs a quick-response grant program, designed for just this purpose. However, until recently researchers were asked to apply for these funds ahead of time in anticipation of the event to be Copyright © 2013 SciRes. 8 T. J. HANEY, J. R. ELLIOTT Copyright © 2013 SciRes. 9 studied (Natural Hazards Center, 2006). Since many research- ers in an affected area will not consider themselves to be disas- ter researchers (or have any interest in disasters) until a catas- trophic event occurs in their locale, many locally affected re- searchers will not have applied for such funding in advance. In the case of Katrina, the National Science Foundation (NSF) commendably sought to ease some of this problem by ear- marking a pool of money for quick-response research, but due to the structural issues discussed above, very few of these funds went to researchers living and working in the affected area. As Table 1 demonstrates, of the roughly $5 million in grant money awarded for social research on Katrina through the NSF Small Grant for Exploratory Research (SGER) program, only about 11 percent was ultimately awarded to researchers from New Orleans, and only about one-quarter was ultimately awarded to researchers from the affected region as a whole (including large sections of Louisiana, Mississippi and Alabama). While not a critique of NSF or any other granting agency, this evidence helps to illustrate how institutional structures work in disaster situations to complicate the research process for local investigators whose sociological imaginations can be undercut by institutional and bureaucratic roadblocks that be- come exacerbated by the very disaster they seek to study. What is lost as a consequence, we believe, is not just research oppor- tunities for local individuals but also local rapport and contex- tual knowledge that can produce higher-quality research and that can provide entrée into privileged spaces that may be un- known or off-limits to outside researchers. In the case of Hur- ricane Katrina, this trove of local knowledge and access was further undercut by local institutional adjustments after the disaster that outside researchers did not face. These adjustments included increased teaching loads, which meant less time for research; suspension of graduate programs, which atrophied lo- cal research cultures and resources; and subsequent relocation of many colleagues to new jobs elsewhere, which often meant that locally initiated research never fully materialized. Further compounding these challenges is also a sense that granting agencies are put off by research that aims to assist in the recovery or carries an advocacy component (Dennis et al., 2006), which may often be precisely the sort of research pur- sued by local researchers who have witnessed the destruction and dislocation first-hand. Some observers even warn that local researchers are potentially too close to the situation to conduct ethical research. For example, Barron et al. (2009) contend that maintaining respondent confidentiality is particularly difficult in disaster research, and that this problem is more severe when the research is conducted by local researchers. As one example, Jacobsen and Landau (2003) note that breaches in confidential- ity occur more frequently when local researchers are involved in the project. We question these findings in the context of Hur- ricane Katrina, however, which affected a large enough popula- tion that researchers, local or non-local, were extremely unlike- ly to know participants outside of the research context. Further, Haney and Barber (2013) argue that local researchers, having experienced the event themselves rather than through the media, may ask different questions, pursue different topics, and may produce research that better reflects and recounts the lived ex- perience of participants. Some research also acknowledges the difficulty of conduct- ing research for those who have been affected themselves. For local researchers, disaster work takes place in the context of locating friends and relatives, repairing homes, negotiating with insurance companies, and dealing with emotional response to the event. As Dennis et al. (2006) write, “for empathetic ‘out- siders’ like us, New Orleans has become a city of bittersweet memories; yet, we do not live with the suffering and grief of losing family members, friends, pets, homes, livelihoods, inde- pendence, and security. We have been able to push the horrific images from our minds and go on with our lives, unlike the citizens of New Orleans and other affected cities, towns, and areas of Louisiana, Mississippi, and Alabama.” Not having to deal with the immediacy of personal concerns is one of the me- chanisms that further privileges non-local researchers following a catastrophic event. Despite such overlapping challenges, however, many local investigators still feel compelled to conduct research in and on disasters of which they unexpectedly become a part. In the case of Katrina, this sociological determination commonly gained strength from personal and professional desires to dispel myths and rumors about the disaster; to correct misinformation circu- lated in the media; and, to further expose social inequalities that contributed to the disaster and threatened fair, equitable recov- ery in its wake. As we engaged these efforts ourselves and set out to survey residents of differently affected neighborhoods of New Orleans, scientific challenges were added to existing in- stitutional ones. Scientific Challenges of Sampling and Survey Methods in a Disaster Setting Shortly after securing modest but vital funding from federal sources (of which respective University Research Offices quick- ly took approximately half for institutional overhead), we began to assemble a small team of researchers affiliated with several New Orleans universities but anchored at Tulane University to study the disaster’s disparate effects on local communities. Table 1. Distribution of the nat io n al s ci e n ce fo u n da t i on ’s small grants for expl oratory research (SGER) funds for social research on hurri c a n e katrina. New Orleans Non-New OrleansLA Non-LA Gulf Coast (LA, MS, AL) Non-Gulf Coast All programs (total $) 635,237 4,908,178 1,225,200 4,318,215 1,489,072 4,054,343 All programs (%) 11.45 88.55 22.10 77.9 26.86 73.14 Social and economic sciences (total $)101,405 1,059,898 161,347 999,956 161,347 999,956 Social and economic sciences (%) 8.73 91.27 13.89 86.11 13.89 86.11 N ote: Source: National science foundation’s (2010) award search interface: http://www.nsf.gov/awardsearch/; Adapted from Haney and Barber (2013). T. J. HANEY, J. R. ELLIOTT The research team further recruited and trained numerous gra- duate and undergraduate students who would participate in data collection and (in some cases) data analysis. Although these efforts began five months after Katrina struck and grew out of an elite university (Tulane), the professional obstacles remained daunting and reveal some of the challenges of conducting re- search both on and in a disaster area where one lives and works. To be sure, collecting data in a disaster zone is never easy. However, being a local investigator means assuming dual roles as both participant and observer, which not only offers local knowledge that can improve research and analysis, but also means that the investigator is often actively involved in the affected area’s recovery as he or she studies it. In our own case, this involvement included repairing our homes, helping others, and re-entering university positions in an environment of ex- treme uncertainty that also included extra teaching and service loads to help keep our departments and universities afloat in a time of crisis. Beyond such heightened personal, social and institutional commitments, however, there are several chal- lenges that are common to investigators researching an area affected by disaster including “protecting research quality and maintaining sufficient sample size” (Henderson et al., 2008), both of which present challenges in a post-disaster context. By way of background, the guiding logic of our own project was to assess neighborhood change by designing a survey in- strument that could capture variation across neighborhoods by socio-demographic and damage characteristics. Given the in- stitutional and practical limitations discussed above, the objec- tive was not to create a representative sample of the entire city. The project was instead designed to focus on six neighborhoods, stratified by level of flood damage and racial composition to see how these two factors conjoined to shape events before, during and after evacuation. Although the research involved a survey instrument, the varying levels of damage among neigh- borhoods necessitated a mixed method design. The research team (aided by a group of trained students) visited four of the tracts (the two less damaged and two moderately damaged neigh - borhoods) on foot, surveying those who had reestablished resi- dence, often in government-provided trailers. For the two heav- ily damaged neighborhoods that remained largely unoccupied, we applied a different, multi-phase strategy that relied on a combination of mail, phone, neighbors, and network referrals. This mixed approach of first studying those who had returned (or who had not left) and then moving to still-displaced indi- viduals or areas is similar to one adopted by Henderson et al.’s (2008) study of local subpopulations. For the first stage, we developed a stratified random design intended to collect socially and spatially representative data from adult householders who had returned or otherwise come to live in the four less damaged neighborhoods under investiga- tion. To start, we randomly selected a single census tract in each neighborhood to simplify data collection, maintain con- sistent spatial and demographic constraints across neighbor- hood samples, and permit comparison with existing census data. Next, we used a street map to number each four-sided block in our selected census tract and then randomly selected twenty- five of these blocks to survey in each tract. For each side of a selected block, we then used site visits to establish the total number of housing units present, counting detached single- family housing as one unit; duplexes as two units; and so forth. Where temporary trailers were already on premises, we in- cluded these structures as proxies for the permanent residences they replaced, and then selected a random unit on each block- side with which to start. As we awaited final institutional approval to enter the field, we used the intervening three weeks to train approximately 150 students at Tulane and Xavier Universities in field survey tech- niques. This training included basic instruction and background information, as well as practice sessions to familiarize students with the “feel” of face-to-face interviewing, which would take place from February through April of 2006 (roughly 6 - 8 months after Hurricane Katrina made landfall). To bolster re- sponse rates, we offered each participant a ten dollar gift card to a major retail chain for completing the survey. Student debrief- ings during and after the study suggest that these gift cards were appreciated but not the primary motivation for respondent participation. Instead, residents reported that they simply wan- ted to tell their stories and were willing or eager to sit through a short survey to do so. After appropriate data entry and cleaning, these efforts yielded a sample of 418 adult respondents across our initial four neighborhood tracts. Table 2 compares data collected from the two least damaged of these neighborhood tracts with tract-level data from the 2000 US Census. This comparison indicates an under-sampling on our part of African-American residents in both areas. In the majority-white neighborhood of the Black Pearl, this under- sampling of black residents occurred by nearly a 25 percent margin. Results also indicate a higher mean age in the sample than in the rest of the tract, an oversampling of residents with a Bachelor’s degree or higher, and an undersampling of those earning less than $20,000 per year. These patterns were dupli- cated in West Riverside, a less-damaged, majority-black neigh- borhood in the city’s Uptown region. Overall, these results in- dicate a systematic under-sampling of less advantaged residents and a corresponding oversampling of more advantaged resi- dents. Since data were collected on foot by visiting occupied residences, this bias represents the ability of more advantaged residents to return home more quickly. It should be noted, however, that the percentage of females and the percentage of respondents over age 65 show a comparatively high degree of representativeness. Table 3 presents data for the two moderately damaged neigh- borhoods in our research design. As we shift to these areas, we would expect the oversampling of more privileged residents to intensify, as these neighborhoods were also surveyed on foot, yet residents faced even greater barriers to return as a result of higher levels of flooding triggered by Hurricane Katrina. Both neighborhoods (Fountainbleau, which is a majority-white neigh- borhood, and Leonidas, which is a majority-black neighbor- hood) generally conform to this expectation by demonstrating an undersampling of African-American residents. Interestingly, however, the Fountainbleau sample is nonetheless highly rep- resentative of the larger neighborhood with regard to other socio-demographic factors such as age, educational attainment, income, and gender. These patterns indicate that, overall, our Fountainbleau sample was surprisingly representative of its encompassing pre-disaster population, with the unique excep- tion of racial minorities. In Leonidas, the moderately damaged majority-black neigh- borhood, results indicate a continued oversampling of more advantaged residents and an undersampling of African Ameri- can residents. Indeed, although more than 75 percent of neighborhood residents were black before Katrina, our sample Copyright © 2013 SciRes. 10 T. J. HANEY, J. R. ELLIOTT Table 2. Comparing survey data (2006) with US census data (2000) for sampled tracts and remaining neighbor- hood tracts in minimally damaged neighborhoods. Sample Sampled tract Remaining tracts in neighborhood Black Pearl (Tract 125) (n = 109) (n = 1772) (n = 1772) % Black 12.5 37.3 -- % Over 65 14.7 18.1 -- Mean age 47.2 34.8 -- % Female 56.5 54.4 -- % Bachelor’s degree or higher 77.1 44.0 -- % Earning less than $20,000 27.8 38.2 -- % Access to vehic l e 80.7 93.2 -- West Rive rside (Tract 106) (n = 95) (n = 1574) (n = 3658) % Black 35.8 54.8 36.1 % Over 65 13.7 1 5.1 13.3 Mean age 45.6 36.3 35.9 % Female 59.6 54.6 53.5 % Bachelor’s degree or higher 45.3 31.9 36.1 % Earning less than $20,000 24.7 4 1.8 34.3 % Access to vehicl e 74.5 88.9 78.0 Note: Source: On-the-ground survey (2006) and 2000 tract-level US census data. Table 3. Comparing survey data (2006) with US census data (2000) for sampled tracts and remaining neighbor- hood tracts in moderately dam aged neighborhoods. Sample Sampled tract Remaining tracts in neighborhood Founteainbleau (Tract 122) (n = 82) (n = 2191) (n = 4549) % Black 4.94 9.9 27.9 % Over 65 9.8 11.5 15.9 Mean age 44.2 32.9 35.9 % Female 54.6 51.9 52.1 % Bachelor’ s degree or higher 63.42 69.5 51.5 % Earning less than $20,000 14.52 16.9 23.8 % Access to vehicl e 75.6 93.1 80.5 Leonidas (Tract 132) (n = 139) (n = 3232) (n = 5721) % Black 51.5 77.1 75.5 % Over 65 16.6 11.3 11.8 Mean age 55.9 34.9 32.0 % Female 60.0 56.2 54.3 % Bachelor’ s degree or higher 48.55 22.4 20.0 % Earning less than $20,000 26.05 37.1 46.5 % Access to vehicl e 66.7 89.9 70.5 Note: Source: On-the-ground survey (2006) and 2000 tract-level US census data. Copyright © 2013 SciRes. 11 T. J. HANEY, J. R. ELLIOTT came back with only 52 percent African-American residents. Similarly, although only 22 percent of pre-disaster residents had a Bachelor’s degree or higher, nearly half of sampled resi- dents reported at least a Bachelor’s degree. Results indicate a substantial undersampling of low-income earners, as well. These patterns not only hint at the difficulties that less-advantaged residents faced in returning to New Orleans after floodwaters subsided but also reveal how these obstacles accrued in largely minority as well as largely white neighborhoods that experi- enced only low to moderate physical damage from t he disaster. At the time, these patterns also provided an early glimpse into how New Orleans was becoming a wealthier and whiter city during early recovery from Hurricane Katrina, owing to the sig- nificant barriers to return faced by those with fewer resources (see Elliott et al., 2010; Fussell, 2007). For the second stage of our survey efforts, we focused on residents from two extremely damaged neighborhoods of New Orleans: the Lower Ninth Ward (a high-poverty, largely Afri- can-American neighborhood) and Lakeview (a low-poverty, largely white neighborhood). A year after Hurricane Katrina struck, both neighborhoods remained almost entirely uninhab- ited; local phone lines were still largely disconnected; and mail forwarding services remained exceptionally slow and income- plete. To overcome these difficulties, we developed a three-phase strategy to collect data from approximately one hundred repre- sentative household heads (21 years of age or older) in each neighborhood. These surveys were administered nine to sixteen months after Hurricane Katrina struck, when recovery efforts elsewhere in the region were well underway, and most educated guesses placed the city’s population at roughly half its pre- Katrina total (Sastry, 2009). To begin, we identified census tracts that comprised each neighborhood of interest, randomly selected one tract, and as in Stage 1, and numbered each four-sided block within the se- lected tract using a local street map. We then randomly selected one address, or housing unit, per randomly selected block-side and matched it to residents’ names and phone numbers using reverse-address information from the latest Polk City Directory for New Orleans. In late May and early June of 2006, approxi- mately nine months after the hurricane hit, we began mailing personally addressed letters describing the survey and offering a $20 gift card to a national retail store for participation. Our hope was that mail-forwarding would ultimately find some sam- pled residents who would then call us to participate in the sur- vey over the phone. Like Henderson et al. (2008), the vast ma- jority of our mailed surveys came back marked as “return to sender.” In July 2006, we began calling phone numbers that we had matched to randomly sampled addresses. Most of these num- bers remained disconnected, but in some cases, new phone numbers were provided by call-forwarding services. In other cases, sampled respondents had established or otherwise Regis- tered new phone numbers accessible on popular internet “white pages” sites, which we searched. If this method failed to pro- duce a working phone number, we used internet phone data- bases (e.g., www.411.com) to collect phone numbers for the four closest neighbors, determined by address. We also utilized such internet databases to help us find those who had relocated to new cities and reestablished phone service. This strategy proved effective for uncommon surnames, but for those with common names such as “Smith,” it proved quite challenging. If we were able to reach a proximate neighbor, and the neighbor was unaware of the whereabouts of the sampled household of interest, we asked the neighbor to participate. Although this latter strategy meant deviating from our original random sample and introducing potential bias into our methods, we figured that having even slightly non-representative data was preferable to having no data at all. Indeed, much like Sastry (2009), we found that it was much more difficult to locate respondents in these heavily damaged neighborhoods than in the less damaged neighborhoods. Although the above strategies worked well for tracking down respondents from the whiter, wealthier Lakeview neighborhood, they were less successful for the Lower Ninth Ward. Thus, for this neighborhood, we were forced to develop and implement a third phase of data collection in November 2006. For this phase, one of our team members (a former neighborhood resident) visited a church that had resumed services in the sampled tract. This collaborator established rapport with the pastor, visited with the congregation of roughly two dozen adults who had returned to the city, and explained the nature of the study. Once rapport had been established, we asked each congregant to pro- vide working phone numbers for adults they knew who had lived in the selected tract when Hurricane Katrina hit. These referees were then contacted by phone and asked to do the same, creating a system of chain referrals. To facilitate these referrals, we provided a $20 gift card to the referrer for all verified refer- als. Research on such “respondent-driven” sampling indicates that after the first or second “link” of referral, such methods can produce samples surprisingly representative of their larger, target populations (Heckathorn, 1997). To assess the representativeness of our data across these dif- ferent methods of data collection, we compared basic demogra- phic characteristics collected from our survey with data com- piled from the 2000 census. Results from these comparesons appear in Table 4 and indicate high representativeness in each tract, albeit with a slight oversampling of higher-income resi- dents in each neighborhood. Moreover, results reveal that our use of referral sampling actually improved data representative- ness among Lower Ninth Ward residents, despite violating best practices for survey sampling. For example, while census data show that 71 percent of households in our sampled tract from the Lower Ninth Ward had access to a vehicle, data collected from respondents in our original random sample indicate that 82 percent of residents had access to a vehicle. Yet, including proximate neighbors in our sample in place of missing random respondents reduces this statistic to 79 percent, which is closer to the census benchmark. In another example, data from respondents in our original sam- ple under-represented the percentage of Lower Ninth Ward residents who earned less than $20,000 per year. This pattern remained the same with inclusion of proximate neighbor. How- ever, since the network referral strategy over-sampled those with low incomes, it provided a correction for the early un- der-sampling. Overall, the implication is that strategies that should have led to less representative data in many ways actu- ally helped to improve its overall quality. In this way and somewhat surprisingly, our experience indi- cates that samples drawn, altered and supplemented for heavily damaged neighborhoods can ultimately end up being more re- presentative than more orthodox sampling strategies in less-da- maged neighborhoods. We suspect that this outcome occurred for several reasons. First, data collection in more damaged areas began six months after data collection in less damaged areas, Copyright © 2013 SciRes. 12 T. J. HANEY, J. R. ELLIOTT Table 4. Comparing survey data (2006) with US census data (2000) for sampled tracts and remaining neighborhood tr acts in heavily damaged neighborhoods. 2006 Surve y 2000 Census Sampling methoda Original sample Proximate neighborNetwork referralFull sampleSampled tract Remaining tracts in neighborhood Lower Ninth (Tract 9.03) (n = 22) (n = 33) (n = 35) (n = 90) (N = 2710) (N = 11,298) % Black 100.0 93.9 97.1 96.6 99.0 99.0 % Over 65 50.0 40.6 28.6 38.2 15.0 13.7 Mean ageb 57.5 55.6 56.0 56.0 33.9 32.4 % Female 81.8 75.8 82.9 80.0 56.3 55.1 % Bachelor’s degree or higher 33.3 16.7 25.7 24.4 6.3 6.9 % Earning less than $20,000 27.8 25.8 55.9* 38.6 49.1 50.8 % Access to vehicle 81.8 68.8 85.7 78.8 71.4 66.7 Lakeview (Tract 56.04) (n = 36) (n = 53) (n = 0) (n = 89) (n = 1878) (n = 7903) % Black 0.0 0.0 -- 0.0 2.5 1.0 % Over 65 13.9 1.9* -- 6.7 19.8 18.4 Mean ageb 49.2 44.2 -- 46.2 38.9 40.3 % Female 69.4 47.2* -- 56.2 55.0 55.20 % Bachelor’s degree or higher 80.6 81.1 -- 80.8 55.9 48.4 % Earning less than $20, 000 0.0 0.0 -- 0.0 18.1 15.8 % Access to vehicle 91.1 92.3 -- 91.9 93.2 91.1 Note: aWe use chi-squared tests to assess whether sub-samples of proximate neighbors and network referrals differ significantly fro m respondents identified in our origi- nally drawn sample. Tests are computed separately for each subsample. Results show, for example, that network referrals from the Lower Ninth Ward do not differ signifi- cantly from originally sampled respondents except with respect to the percentage of families earning less than $20,000. Here, use of network referrals appears to produce a more representative sample of the encompassing tract and remaining neighborhood area than res pondents from the orig inal rand om sample; bFor age, we use a t-test rather than a chi-squared test to assess subsample differences; *p < 0.05. Table adapted from Elliott, Haney, and Sams-Abio dun (2010). giving residents more time for return to the city, if not their own neighborhoods. Second, data collection took place by phone rather than door-to-door, allowing recruitment to occur across large distances. Third and relatedly, our strategy of using pro- ximate neighbors (in both neighborhoods) and network referrals (in the Lower Ninth Ward) allowed us to better access residents who had not returned. Discussion Although research on and in contexts of extreme disaster presents many challenges, especially for local investigators, our own efforts demonstrate that determination can generate a rea- sonably representative sample of pre-disaster residents for pur- poses of social research. Normally, an in-person survey re- sponse rate of 60 to 70 percent is considered quite good (Sin- gleton and Straits, 1999). By phone or e-mail, response rates are usually appreciably lower (Cook et al., 2000; Curtin et al., 2005). Yet, in our Katrina study, we found that response rates even in heavily damaged neighborhoods topped 90 percent. The real challenge was to locate those in our sample, but once we located these potential respondents (or their proxies in more damaged neighborhoods) nearly everyone agreed to participate. This finding is echoed by Henderson et al. (2008) who report that in their own research, “having a sufficient sample size was threatened because the team found it difficult to contact poten- tial study participants,” not because those who were contacted declined to participa te . It is worth noting here that we did offer a small incentive in the form of a retail gift card to help offset anticipated problems with sample size. However, many respondents turned down the gift card. Others expressed very clearly to us that, although they appreciated the incentive, their desire in participating was to help us create knowledge about the event—knowledge that they hoped would draw attention to the plight of New Orleanians and would help to ensure that an event like this one never un- folded again the same way. So, while disasters such as Hurri- cane Katrina present methodological and scientific challenges, as described above, they also present unique opportunities for participants to connect with the broader research process in meaningful ways. In this sense, Tomaselli et al. (2005) explain, research participants often “want their reciprocal dues—from academics, journalists, photographers and others who take, but do not always reciprocate in ways that they can understand or appreciate.” Our participants clearly considered these reciprocal dues when making decisions about whether to participate in our study, much more so than considering the explicit if modest financial incentive we offered. As we continued to turn our research gaze inward, we also learned the importance of establishing trust between researcher and participant. Although previous research expresses concern that locally affected populations may experience survey fatigue Copyright © 2013 SciRes. 13 T. J. HANEY, J. R. ELLIOTT or may become burdened by efforts to collect data from them (Fleischman and Wood, 2002), our participants expressed not fatigue, but reasoned skepticism. They mentioned their worry that researchers (especially non-local researchers) were ex- ploiting residents’ pain for personal or professional gain. As a result, participants would often ask us to demonstrate our own local identities by showing them a business card, telling them our addresses, answering questions about local geography, or otherwise demonstrating to them that we lived in the area and were affected by Katrina as well. Some of the earliest disaster research demonstrates that locally affected people are hesitant to share their experience with outsiders out of a belief that out- siders cannot possibly understand the disaster experience (Erikson, 1976; Hoffman, 1999), and our findings affirm this sensibility. In many instances, until we could demonstrate that we too suffered loss and trauma from the disaster, many New Orleanians were hesitant to speak with us. One additional lesson learned from our determination relates to the cathartic effect of research among those who participated. As numerous researchers have discovered, those who experi- ence stressful or traumatic events often gain benefits from writing or speaking about those events (Pennebaker et al., 1990; Smyth & Pennebaker, 1999). Anecdotal reflections on our re- search process are consistent with this observation. Some par- ticipants enjoyed talking with the researchers so much that a simple 20-minute survey often took more than 90 minutes to complete, as each closed-ended question became a new story to tell. Many participants both laughed and wept (sometimes, so did the researchers) and the vast majority of participants thank- ed us for asking them to participate. Although Henderson et al. (2008) found that nine months after the storm “residents were less inclined to discuss the topic, introducing threats to internal validity, such as history and maturation,” we found participants eager to talk at length about Katrina and its aftermath. These conversations revealed a great deal of concern, fear, anger, and self-reflection. Lastly, and perhaps most importantly, research such as ours demonstrates that although scientific “best practices” work during routine situations, disasters such as Katrina can become non-routine in just about every imaginable way. Consequently, disasters present unique challenges and require innovative solu- tions, solutions that often do not conform to the ideal sampling methods or instrument design strategies that might be appropri- ate during non-disaster situations. In our research, we needed to bend some of the textbook rules in order to complete our re- search, yet by doing so carefully and deliberately our final sample exhibited a surprisingly high degree of representative- ness. If there was one consistent pattern, it was the slight over- representation of older residents and more advantaged residents (in terms of education, income, and race/ethnicity). This finding indicates that future research should continue to strive to find ways of locating and including the least advantaged residents of and from affected a reas. Conclusion As the present study shows, local researchers occupy both a privileged and disadvantaged position in times of disaster. By experiencing the event themselves, they gain unique access to the collective experiences of those involved. They also possess useful knowledge of the local history, geography, and politics of the affected area—all of which can be tapped as resources in the drive for higher quality research. At the same time, con- ducting research inside a disaster area and within severely com- promised local bureaucracies can also present numerous barri- ers to entry and can curtail local disaster research before it gets very far along. The present study sought to illuminate some of these barriers, especially those encountered disproportionately by local inves- tigators. We contend (even after leaving the disaster zone we studied) that the disaster research community should begin to take more explicit steps to foster local research by those in affected regions by first becoming more aware of the multidi- mensional challenges we describe here and second, by taking collective steps to address them. Such steps may include pre- arranging proxy institutional review boards, earmarking a cer- tain portion of grant money for those in the affected region, or facilitating partnerships and collaborations between those in affected regions and those working from unaffected institutions with continued access to normal resources and processes. Re- gardless, we believe that the relative success of our own efforts, however unorthodox, speak to the value of further enabling more much instances of such sociological determination. Also and more specifically , we also submit that a blended strategy of on-the-ground surveys for returnees and telephone surveys for those still displaced provides one of the most instructive exam- ples to date of how researchers can and should collect data on displaced populations in a timely fashion, despite concerns generated by “best practices” under normal conditions. In conclusion, we hope that by documenting these challenges and opportunities, we may contribute important knowledge about how disasters can generate methodological innovation and, perhaps more importantly, how the most rewarding re- search may occur when participants themselves feel invested in and committed to the research process. In this broader sense, we believe the ethical imperative of disaster research should be to produce knowledge that ultimately mitigates vulnerability and speeds recovery. Our work is further motivated by the hope that sociologists can play an important role in mitigating vul- nerability to future disasters and catastrophic events. With the advent of vulnerability science and growing public awareness of disasters, we believe that sociologists can and should take center stage in these efforts by informing others of the chal- lenges faced in conducting such research and by collecting data in and around such events through an ongoing blend of socio- logical imagination and determination. Acknowledgements Some of the research discussed in the present study received support from the National Science Foundation (Award #0554818). We thank residents of New Orleans for their generous partici- pation in all phases. An earlier version of this paper was presented at the 2012 International Conference on Methods for Surveying and Enu- merating Hard to Reach Populations, a Conference of the Ame- rican Statistical Association, New Orleans, LA. We thank con- ference participants for helpful feedback. REFERENCES Barron, A. C. Y., Barrett, E. J., & Martinez-Cosio, M. (2009). Ethical issues in disaster research: Lessons from hurricane katrina. Popula- tion Research and Policy Review, 28, 93-106. Copyright © 2013 SciRes. 14 T. J. HANEY, J. R. ELLIOTT Copyright © 2013 SciRes. 15 doi:10.1007/s11113-008-9112-7 Baehr, P. (2005). Social extremity, communities of fate, and the soci- ology of SARS. European Journal o f Sociology, 46, 179-211. doi:10.1017/S000397560500007X Cook, C., Heath, F., & Thompson, R. L. (2000). Analysis of response rates in web- or internet-based surveys. Educational and Psycholo- gical Measurement, 60, 821-836. doi:10.1177/00131640021970934 Curtin, R., Presser, S., & Singer, E. (2005). Changes in telephone sur- vey non-response over the past quarter centu ry. Public Opinion Quar- terly, 69, 87-98. doi:10.1093/poq/nfi002 Dennis, M. R., Kunkel, A. D., Woods, G., & Schrodt, P. (2006). Mak- ing sense of New Orleans flood trauma recovery: Ethics, research design, and policy considerations for future disasters. Analyses of So- cial Issues and Public Policy, 6, 191-213. doi:10.1111/j.1530-2415.2006.00107.x Durkheim, E. (1951). Suicide: A study in sociology. New York: The Free Press. Elliott, J. R., Haney, T. J., & Sams-Abiodun, P. (2010). Limits to social capital: Comparing network activation in two New Orleans neigh- borhoods devastated by Hurricane Katrina. The Sociological Quar- terly, 51, 624-648. doi:10.1111/j.1533-8525.2010.01186.x Erikson, K. (1976). Everything in its path: Destruction of community in the Buffalo Creek flood. New Yor k: Si mon and Schuster. Fleischman, A. R., & Wood, E. B. (2002). Ethical issues in research involving victims of terror. Journal of Urban Health: Bulletin of the New York Academic of Medicine, 79, 315-321. doi:10.1093/jurban/79.3.315 Fussell, E. (2007). Constructing New Orleans, constructing race: A po- pulation history of New Orleans. Journal of American History, 94, 846-855. doi:10.2307/25095147 Haney, T. J., & Barber, K. (2013). Reconciling academic objectivity and subjective trauma: The double consciousness of sociologists who experienced Hurricane Katrina. Critical Sociology, 39, 102-122. Haney, T. J. (2007). Disaster and the irrationality of “rational” bu- reaucracy: Daily life and the continuing struggles in the aftermath of Hurricane Katrina. In D. A. Hidalgo, & K. Barber (Eds.), Narrating the storm: Sociological stories of hurricane Katrina (pp. 128-138). Newcastle: Cambridge Scholars Press. Heckathorn, D. (1997). Respondent-driven sampling: A new approach to the study of hidden populations. Social Problems, 44, 174-199. doi:10.2307/3096941 Henderson, T. L., Sirois, M., Ch en, A. C. , Air riess, C., Swanson , D. A., & Banks, D. (2008). After a disaster: Lessons in survey methodology from Hurricane Katrina. Population Research Policy Review, 28, 67-92. doi:10.1007/s11113-008-9114-5 Hoffman, S. M. (1999). The regenesis of traditional gender patterns in the wake of disaster. In A. Oliver-Smith, & S. M. Hoffman (Eds.), The angry earth: Disaster in anthropological perspective (pp. 174- 191). New York: Routledge. Jacobson, K., & Landau, L. B. (2003). The dual imperative in refugee research: Some methodological and ethical considerations in social science research on forced migration. Disasters, 27, 185-206. doi:10.1111/1467-7717.00228 Klinenberg, E. (2002). Heat wave: A social autopsy of disaster in Chi- cago. Chicago, IL: University of Chi cago Press. Knack, J. M., Chen, Z., Williams, K. D., & Jensen-Campbell, L. A. (2006). Opportunities and challenges for studying disaster survivors. Analyses of Social Issues and Public Policy, 6, 175-189. doi:10.1111/j.1530-2415.2006.00116.x Mills, C. W. (1956). The sociological imagination. New York: Oxford University Press. Molotch, H. (2006). Death on the roof: Race and bureaucratic failure, social science research council’ s un derstanding katrina we bs it e. http://understandingkatrina.ssrc.org/Molotch/ Natural Hazards Center (2006). Learning from catastrophe: Quick res- ponse research in the wake of hurricane katrina. Boulder, CO: Natu- ral Hazards Center. Pennebaker, J. W., Colder, M., & Sharp, L. K. (1990). Accelerating the coping process. Journal of Personality and Social Psychology, 58, 528-537. doi:10.1037/0022-3514.58.3.528 Sastry, N. (2009). Tracing the effects of hurricane katrina on the popu- lation of New Orleans: The displaced New Orleans residents pilot study. Sociological Methods and Research, 38, 171-196. doi:10.1177/0049124109339370 Singleton, R. A., & Straits, B. C. (1999). Approaches to social research (3rd ed.). New York: Oxford University Press. Smyth, J. M., & Pennebaker, J. W. (1999). Sharing one’s story: Trans- lating emotional experiences into words as a coping tool. In C. R. Snyder (Ed.), Coping: The psychology of what works (pp. 70-89). New York: Oxford University Press. Snider, L. (2004). Resisting neoliberalism: The poisoned water disaster in Walkerton, Ontario. Social and Legal Studies, 13, 265-289. doi:10.1177/0964663904042554 Steinberg, P. (2008). What is a city? Rethinking the urban after hurri- cane katrina. Athens, GA: University of Georgia Press. Tomaselli, K., McLennan-Dodd, V., & Shepperson, A. (2005). Re- search to do, results to sell: Enabling subjects and researchers. Soci- ety in Transition, 36, 24-27. doi:10.1080/21528586.2005.10419126 Weber, M. (1946). From Max Weber: Essays in sociology. New York: Oxford University Press. |