Methodology | Pew Research Center

The American Trends Panel study methodology

The American Trends Panel (ATP), produced by Pew Analysis Middle, is a nationally consultant panel of randomly picked U.S. older people. Panelists take part by means of self-administered net surveys. Panelists who do not have internet obtain at house are provided with a tablet and wireless internet connection. Interviews are done in equally English and Spanish. The panel is getting managed by Ipsos.

Information in this report is drawn from the panel wave done Aug. 31 to Sept. 7, 2020. A overall of 9,220 panelists responded out of 9,810 who had been sampled, for a reaction price of ninety four{bf9f37f88ebac789d8dc87fbc534dfd7d7e1a7f067143a484fc5af4e53e0d2c5}. This does not include two panelists who had been eradicated from the details thanks to extremely large charges of refusal or straightlining. The cumulative reaction price accounting for nonresponse to the recruitment surveys and attrition is five{bf9f37f88ebac789d8dc87fbc534dfd7d7e1a7f067143a484fc5af4e53e0d2c5}. The crack-off price among the panelists who logged on to the study and done at least one merchandise is one{bf9f37f88ebac789d8dc87fbc534dfd7d7e1a7f067143a484fc5af4e53e0d2c5}. The margin of sampling mistake for the entire sample of 9,220 respondents is plus or minus one.7 percentage factors. The study is weighted to be consultant of the U.S. grownup inhabitants by gender, race, ethnicity, partisan affiliation, training and other groups.

Additional data about this wave of the American Trends Panel can be found here. 

Content analysis

Researchers done their analysis on a collection of 9,220 raw responses to the adhering to open-ended study queries:

Prior to we complete the study, we’re intrigued in acquiring a deeper understanding of how individuals have been enduring the coronavirus outbreak, and how it might have impacted them in equally negative and beneficial ways. Be sure to take a moment to replicate on how the outbreak has affected your individual lifetime, and then respond to the upcoming two queries in as considerably depth as you can.

1. Imagining about how items have been heading in your lifetime considering the fact that the beginning of the coronavirus outbreak, in what ways, if any, have items been challenging or hard for you personally?
2. Imagining about how items have been heading in your lifetime considering the fact that the beginning of the coronavirus outbreak, what, if just about anything, have been some beneficial aspects of the situation for you personally?

The two queries had been requested in a random order. Over-all, 84{bf9f37f88ebac789d8dc87fbc534dfd7d7e1a7f067143a484fc5af4e53e0d2c5} of respondents (8,143) provided an respond to to one or equally of the queries (the sixteen{bf9f37f88ebac789d8dc87fbc534dfd7d7e1a7f067143a484fc5af4e53e0d2c5} of respondents that skipped equally queries are not provided in this analysis). From these responses, researchers drew a 50 {bf9f37f88ebac789d8dc87fbc534dfd7d7e1a7f067143a484fc5af4e53e0d2c5}-sample of four,071 responses to evaluate and evaluate their written content. Responses that had been published in Spanish – three{bf9f37f88ebac789d8dc87fbc534dfd7d7e1a7f067143a484fc5af4e53e0d2c5} of the sample – had been device translated and examined by researchers to affirm their interpretability.

Following drawing exploratory samples to establish widespread themes in the responses, researchers created a codebook to evaluate the distinctive varieties of beneficial and negative adjustments. Coders had been revealed respondents’ solutions to equally the beneficial and negative queries facet-by-facet, prefixed by “POSITIVE:” and “NEGATIVE:” labels to indicate the prompt that elicited every reaction. Coders had been instructed to first indicate irrespective of whether every reaction mentioned any beneficial and/or negative adjustments, and then to mark individuals that fell into any of six distinctive groups: associations free time or pursuits perform situation economics actual physical or mental overall health and society, politics, basic safety precautions or rules. Added guidance and rules had been provided to help coders make their choices. Two in-residence researchers iteratively drew and coded samples and expanded the codebook rules until eventually suitable inter-rater dependability was achieved.

Codebook

We requested two queries on a modern study, one about negative impacts of the pandemic, and a further about unexpectedly beneficial outcomes. The ‘POSITIVE’ and ‘NEGATIVE’ labels below indicate how a respondent answered every dilemma, but they could have mentioned beneficial items when answering the negative dilemma, and vice versa. Be sure to use the labels as a information – if they do not supply additional context for their reaction, you can presume that it’s answering the dilemma that was requested. If they give more particulars, use that to help make your determination.

• “NEGATIVE: more time at house: – presume this is a negative point
• “NEGATIVE: perform has been hard, but I have more time at home” – they mention perform as a negative point, but time at house is a beneficial point

Only mark a thing if it is a improve or distinction in their lifetime if they mention a thing that has stayed the similar (“My family has stayed healthier,” “I’m still broke and unemployed”) do not depend it.

Be aware: the Beneficial and Destructive prompt markers in the responses are guides. Just due to the fact respondents wrote a thing, does not necessarily mean what they wrote matched the prompt – go through the responses thoroughly when answering this. Any beneficial adjustments mentioned in possibly prompt counts as specifics, even if they aren’t captured by any of the checkbox groups below.

Beneficial Variations: Does this reaction mention any specific beneficial adjustments in the course of the pandemic?

Associations: top quality/frequency of social interactions (themselves)

• Expending more time with family/buddies/neighbors
• Developing closer to other people/establishing better associations
• Experiencing not possessing to socialize as considerably
• DO NOT Rely church until a social element is mentioned
• DO NOT Rely shelling out time with animals

No cost TIME/Functions: No cost time, pursuits or hobbies

• Possessing more free time/more time to do items all-around the residence
• Currently being capable to do more items/shelling out more time executing items at house
• Buying up new pursuits or hobbies
• Learning new items
• Much less time invested on unfulfilling pursuits like driving (Work Condition)
• Slowing down/possessing a slower pace of lifetime/getting capable to loosen up more (Physical/Psychological Health and fitness)
• Experiencing shelling out (more) time at house
• Possessing more time for personal reflection (Physical/Psychological Health and fitness)
• DO NOT Rely socializing or observing/traveling to individuals until it mentions a specific exercise like ‘going out’ or ‘traveling’
• USE THE PROMPTS AS CONTEXT: ‘more time at home’ below the Beneficial prompt would depend, but below the Destructive one it would not

Work Condition: top quality or availability of career/perform/career (themselves or individuals they know)

• Currently being capable to perform from house
• Possessing an enhanced perform situation
• Possessing a better perform-lifetime harmony
• Discovering a new career
• Not possessing to go into the office environment
• Not possessing a commute (No cost TIME/Functions)
• Not possessing to perform as considerably
• Accomplishment maintaining their organization
• Utilized the situation as an option to retire or do a thing distinctive

ECONOMICS: Funds, revenue or economics (in any sense)

• Preserving more revenue/shelling out considerably less revenue
• Acquiring stimulus or unemployment funds/possessing an enhance in cash flow
• Emotion like the economic climate is executing well

Physical/Psychological Health and fitness: mental or actual physical overall health results, conditions or behaviors (themselves or individuals they know)

• Slowing down/possessing a slower pace of lifetime/getting capable to loosen up more (No cost TIME/ Functions)
• Emotion happier or more comfortable/considerably less pressured/more written content
• Possessing more time for personal reflection (No cost TIME/Functions)
• Improved non secular lifetime/higher spirituality
• Higher appreciation for items that are vital/possessing higher clarity
• Emotion a renewed sense of gratitude for what they have
• Shelling out more consideration to their overall health/getting in better overall health/dropping excess weight
• Performing healthier pursuits like training, strolling, mountaineering, consuming better, acquiring more slumber
• Performing less unhealthy pursuits/ingesting considerably less

Modern society/POLITICS/Safety Safety measures/Principles: sights on politics, social conditions, basic safety precautions like putting on masks, and/or rules and polices

• Approves of federal government reaction to the pandemic/explicitly expresses support for basic safety precautions/rules/polices/constraints
• Expresses gratification with or support for specific political teams/sights
• Beneficial developments in society like: individuals practising better hygiene, adhering to basic safety precautions, considerably less air pollution, less targeted visitors, sites getting considerably less crowded, beneficial environmental developments
• Expresses gratification with greater awareness of social/political concerns/the ‘truth’ getting exposed
• DO NOT Rely uncomplicated mentions of adhering to basic safety precautions/rules they will need to evidently mention it as a beneficial point that they support
• DO NOT Rely appreciating necessary personnel

Destructive Variations: Does this reaction mention any specific negative adjustments in the course of the pandemic?

Associations: top quality/frequency of social interactions

• Not getting capable to socialize, see or visit individuals/diminished social lifetime/less social interactions
• Deteriorating associations/greater conflict
• Lack of actual physical get hold of with other people
• The dying of family associates or buddies (Physical/Psychological Health and fitness)
• The specific emotions of isolation or loneliness (Physical/Psychological Health and fitness)
• DO NOT Rely church until a social element is mentioned

No cost TIME/Functions: No cost time, pursuits or hobbies

• Not getting capable to do items they employed to do or had been arranging on executing (e.g. procuring, church, vacation, gatherings, holidays, appointments)
• Emotion pent up/possessing cabin fever/getting trapped at house
• Possessing as well considerably free time/experience bored
• Not possessing ample free time
• Not getting capable to go to church/missing church
• DO NOT Rely not getting capable to socialize or see/visit individuals until it mentions a specific exercise like ‘going out’ or ‘traveling’
• USE THE PROMPTS AS CONTEXT: ‘more time at home’ below the Destructive prompt would depend, but below the Beneficial one it would not

Work Condition: top quality or availability of career/perform/career (themselves or individuals they know)

• Possessing to perform from house
• Career insecurity/experience unsure about their employment
• Getting rid of their career/complaining about not possessing a career/not ample perform/possessing their hrs or spend lower/possessing considerably less perform out there/acquiring furloughed (Financial)
• Possessing to go to perform (getting an necessary worker)
• Additional tension at or about perform/experience unsafe at perform
• Currently being pressured to retire
• DO NOT Rely standard unemployment or generic references to individuals dropping their work opportunities or getting out of perform (that would be Financial rather)
• DO NOT Rely not possessing ample revenue until it mentions a spend reduction or decline of cash flow

ECONOMICS: Funds, revenue or economics (in any sense)

• Getting rid of their career/complaining about not possessing a career/not ample perform/possessing their hrs or spend lower/possessing considerably less perform out there/acquiring furloughed (Work Condition)
• General unemployment or generic references to individuals dropping their work opportunities or getting out of perform
• Destructive impacts on the economic climate or modest companies/organization closing/standard poor overall health of the economic climate
• Expending more revenue
• Possessing considerably less cash flow/paycheck reductions
• Operating out of revenue
• Additional fiscal uncertainty
• Merchandise shortages or shortage
• DO NOT Rely references to ‘job insecurity’ or other perform-linked concerns until there is a distinct and obvious financial impact (i.e. career decline or one of the other specific topics mentioned over)

Physical/Psychological Health and fitness: mental or actual physical overall health results, conditions or behaviors (themselves or individuals they know)

• Not getting capable to get healthcare/preventing healthcare or physicians
• Currently being in even worse overall health than they had been in advance of/attaining excess weight
• Performing unhealthy pursuits like ingesting more
• Performing less healthier pursuits like training, strolling or mountaineering
• Taking in inadequately/attaining excess weight
• Not getting capable to work out or go to the health and fitness center
• The dying of family associates or buddies (Associations)
• Obviously expresses concern or worry about getting at-chance for COVID or acquiring COVID/apprehensive about pre-current affliction
• Not experience risk-free
• Somebody specific (themselves or somebody they know) getting or acquiring unwell or dying, COVID or or else
• The specific emotions of isolation or loneliness (Associations)
• Emotion mentally taxed/experience like they simply cannot love lifetime
• Describes experience pressured, frustrated, concerned, indignant, sad, disappointed or apprehensive
• DO NOT Rely statements that sound indignant or frustrated and many others they will need to mention a mental/emotional condition or experience a specific way emotionally

DO NOT Rely statements that describe careful habits (e.g. ‘we are getting cautious’) or uncomplicated mentions of pre-current conditions (e.g. ‘we are at risk’) until they particularly mention getting apprehensive, anxious, concerned, and many others.

Modern society/POLITICS/Safety Safety measures/Principles: sights on politics, social conditions, basic safety precautions like putting on masks, and/or rules and polices

• Emotion like they are getting lied as well/not trusting the ‘truth’
• Expresses disapproval of specific political teams/sights
• Expresses irritation with or skepticism of safety precautions/masks/rules/polices/constraints/lockdowns/curfews
• Disapproves of federal government reaction to the pandemic
• Destructive developments in society like individuals not adhering to basic safety precautions or not using the virus severely, more partisan conflict, individuals getting rude
• Destructive sights of specific public figures or institutions like politicians, the media, or scientists
• Emotion stigmatized or judged for their sights
• Currently being sequestered/getting on obligatory lockdown
• Imagining the virus is overblown and that items need to get back again to typical
• Consists of mentions of basic safety precautions like masks and social distancing or mention of rules/constraints/lockdowns/curfews if they show up below the Destructive prompt, until additional context indicates or else

• DO NOT Rely generic reference to ‘quarantine’ if they are just employing it to refer to the standard pandemic situation
• Last inter-rater dependability between the two in-residence coders, as calculated by Cohen’s Kappa, is described below for a ultimate sample of 200 responses. The frequency of every class and the dimension of the sample was adequate to decide with 95{bf9f37f88ebac789d8dc87fbc534dfd7d7e1a7f067143a484fc5af4e53e0d2c5} confidence that the correct Kappa values for every class are at least .7.

Last inter-rater dependability between the two in-residence coders, as calculated by Cohen’s Kappa, is described below for a ultimate sample of 200 responses. The frequency of every class and the dimension of the sample was adequate to decide with 95{bf9f37f88ebac789d8dc87fbc534dfd7d7e1a7f067143a484fc5af4e53e0d2c5} confidence that the correct Kappa values for every class are at least .7.

Prior to conducting the entire written content analysis, researchers divided the responses into two teams: individuals that contained potentially sensitive personal particulars, and individuals that did not. In-residence researchers then coded 360 responses that did incorporate this sort of data, and employed Amazon’s Mechanical Turk service to code the remaining three,711 that did not. A few Mechanical Turk personnel coded every reaction, and their choices had been collapsed employing a two-out-of-three consensus threshold to deliver coding choices equivalent to individuals of our individual in-residence researchers.

To evaluate this inter-rater dependability, in-residence researchers coded a subset of the responses that had been despatched to Mechanical Turk. A critique of initial effects showed promising dependability but also indicated that Mechanical Turk personnel had been frequently missing some of the more nuanced coding rules. These mistakes had been reliable and obvious for illustration, Mechanical Turk personnel had been properly determining negative adjustments linked to actual physical overall health, but appeared to be normally missing responses that explained negative mental health adjustments this sort of as melancholy and tension. Following noticing these designs, researchers had been capable to acquire a handful of sample-matching rules to make corrections to the Mechanical Turk workers’ choices centered on irrespective of whether specific key terms (e.g., “depress” or “stress”) had been mentioned in the beneficial or negative section of every response’s text.

Following coding a ultimate expanded sample of 400 responses, researchers established that this mixed Mechanical Turk and sample-matching strategy generated effects equivalent to our individual in-residence choices, with the exception of beneficial adjustments linked to society, politics, basic safety precautions and rules. 

For the ultimate analysis, researchers mixed the 360 responses that had been coded in-residence with the three,711 responses that had been coded by Mechanical Turk personnel, for a ultimate sample of four,071 responses. Fully 95{bf9f37f88ebac789d8dc87fbc534dfd7d7e1a7f067143a484fc5af4e53e0d2c5} of these responses mentioned at least one beneficial or negative improve 67{bf9f37f88ebac789d8dc87fbc534dfd7d7e1a7f067143a484fc5af4e53e0d2c5} mentioned at least one of every, 22{bf9f37f88ebac789d8dc87fbc534dfd7d7e1a7f067143a484fc5af4e53e0d2c5} mentioned one or more negative adjustments but no beneficial adjustments, and five{bf9f37f88ebac789d8dc87fbc534dfd7d7e1a7f067143a484fc5af4e53e0d2c5} mentioned one or more beneficial adjustments but no negative types.

Demographic discrepancies

All described demographic discrepancies had been verified to be statistically important with 95{bf9f37f88ebac789d8dc87fbc534dfd7d7e1a7f067143a484fc5af4e53e0d2c5} confidence employing logistic regressions that isolated the associations between demographic factors and a respondent’s probability of mentioning a beneficial or negative improve in every class. The regression types provided variables for race/ethnicity, political celebration identification, training, age, urbanicity, parenthood, cash flow, gender, geographic location and language (English vs. Spanish), and also managed for the all round length of every responses employing log-scaled word counts.