Chapter 8: Survey Questionnaire §

(F) Day of the week: Tuesday Class: IS303 Created Time: December 22, 2020 2:41 PM Database: Class Notes Database Date: December 22, 2020 2:41 PM Days Till Date: Passed Last Edited Time: June 9, 2021 10:42 AM Type: Presentation Notes

1. Surveys §

Survey: a data collection tool in social science, posing questions to respondents

• It’s versitile: flexible

Two forms of Survey

• Interview: questions posed person to person (phone/in person)
• Questionnair: self administered question answering

Problems of respondent cooperation §

• Respondense Rate: percentage of people who complete the survey
• Social Desirability: error caused by people answering
  • in a socially appropriate way, what is expected instead of reality
  • lie to make themselves look good, and hide bad things

2. The way we word things §

Operational

• Concepts are measured through questions posed to subjects
• Careful exact wording of questions aiming for the right variable
  • Sensitivity: Welfare vs Poor, End Pregnancy vs Abortion

2. The Rules §

2.1. Questions should be clear of ambiguity §

Use precise language in questions: not subjective wordings

Do you exercise regularly? vs How many times do you exercise per week?

2.2. Use common language §

• Don’t use technical words: field specific jargon will only confuse people, use everyday words
• Don’t use abbreviations: field specific abbreviations isn’t commonly known and may be mis understood.

2.3. Use neutral language §

• Don’t use emotional facts, language, or phrases that seek empathy, fear, or pity
• Causing emotions could skew/bias an answer to the emotion instead of the issue itself

2.4. Use Simple to Answer Questions §

The double barrel question: to ask two or more questions in disguise as one question.

Ex: How do you rate police response time to emergency and non-emergency calls?

3. The Structure of Questions §

3.1. Closed Ended Questions §

Questiosn for you to choose from a number of options available

• Advantage: Easier to answer for respondents
  • Save time to analyze
• Disadvantage: not exactly fit for all respondents
  • Options might not

Ex: Nominal & Ordinal level of measurement: provide fixed choices

Do you rent DVDs?

Traits of Closed Ended

• Exhaustivity: present all relevant response as a option to respondents
• Mutual exclusivity: each response options must not overlap each other
• Nominal level measures must be mutually exclusive and exhaustive

Level of agreement could be from 2 to 6

3.2. Open Ended Questions §

Questions requiring repondents to give their own answer

• Advantage: better at getting clear and personalized answers from respondent
  • Uncover surprising element
• Disadvantage: lower response rate from respondents as it’s longer to do
  • Unable to standardize
  • Difficult to code or analyze data

In the last two weeks, how many times have you rented dvds?

4. Putting it together §

Demand repondents to supply requested informations.

How? Certain questionnair formats could help

4.1. Cover Letter or Introduction §

to sell your research, respondents to be interested to help

• Importance: how much it could impact society, that they are important part
• Legitimacy: how we will follow up and create meaningful results
• Confidentiality: anonimity and privacy should be addressed

Cover Letter

• Personalize: address name, catagories, or status
• Have personal signature of researchers
• Contain contact information
• Put cover letter on questionnair

4.2. Primacy Effects §

“Item at the beginning of a list will stay in people’s long term memory longer”

Don’t start with a

• demographic question (age, gender, edu)
• openended-questions

Do start with

• Interesting opinion
• Attitude questions: non-sensitive safe questions

4.3. Sequencing §

Ordering your questions properly by catagorizing will help resondents not get confused

4.4. Logical Flow §

Earlier questions could influence answer to later questions

• Grouping by time order
• Catagorizing
• Bring thought provoking question about the topic before asking the real question

How to deal with

• Determine sensitivity of the order
• Pre-test with trail respondents

4.5. What to ask and what not to §

Filter Question: question that determine if repondent are supposed to skip or answer a question

Convert long questionnairs to shorter ones.

Why: The longer the questionnair, the less the response rate

How: a questionnair shouldn’t be more than 40 minuites long

The correct format:

• not to be crammed
• give enough answer space

4.6. Formatting §

Best layout to put questions

Resonse Set: questions that are worded the same into a pattern that could cause error

Mis match questions to

5. Pre-testing §

think of format, logical flow of survey and tweak according to feedback

Pre test methods:

5.1. Cognitive Interview §

Think Aloud §

• Repondents are asked to verbalize their answers
• non-directive and open-probing

Directive and Probing §

• Asking for open ended questions

6. Challenges to Mailed Questionnairs §

Cover Letter: make easier response, that it’s important enough to fill and send back

Make returning Easy and attractive: make short surveys, correct time, incentive rewards

More use of online surveys

Online Survey §

Advantage:

• Quick and convenient to fill in
• No need to use documents or transportation
• Reach wider audience

Disadvantage:

• Biased towards respondents who have tech devices, and internet access
• No accessible to the poor, old people, rural areas
• Respondents might not care much about answering honestly