Monday, January 23, 2012

Desinging of Questionnaire

General Guidelines

1. What is a questionnaire?

An instrument (form) to

collect answers to questions

collect factual data

Gathers information or measures

A series of written questions/items in a fixed, rational order

2. Why using a questionnaire?

A well designed questionnaire:

Gives accurate and relevant information to your research question

Minimises potential sources of bias

Will more likely be completed

3. Advantages of questionnaires

Can reach a large number of people relatively easily and economically

Provide quantifiable answers

Relatively easy to analyse

4. Disadvantages of questionnaires

Provides only limited insight into problem

Limited response allowed by questions

Maybe not the right questions are asked

Varying response

Misunderstanding/misinterpretation

Need to get it right first time

Hard to chase after missing data

5. Types of Questionnaire

· Face to face

· Telephone

- Interviewer administered

· By post

· e-mail/ internet

- Self administered

6. What makes a well designed questionnaire?

Good appearance easy on the eye

Short and simple

Relevant and logical

Ø High response rate

Ø Easy data summarisation and analysis

7. Basic Rules

On first page

- Return address

- Study title in bold

On all pages

- Identifying mark/unique identifier

- Numbered items

- Page numbers

Directions in bold

Self-addressed envelope!!

8. Questionnaire Introduction

- Covering letter/ interview introduction

- Who you are/ you work for

- Why you are investigating

- Where you obtained the respondent’s name

- How and where you can be contacted

- Guarantee of confidentiality

- Length of interview (be honest)

Uusefulness of study should be clear to all respondents

9. Questionnaire Design Steps

- Decide what information is needed

- Search for existing questions

- Focus groups

- Draft new questions/revise existing ones

- Sequence the questions

- Get peer evaluation

- Revise and test on self/co-workers

- Think-aloud interviews

- Revise/eliminate questions

- Prepare interviewer instructions for pilot test

- Pilot test (10-20 cases)

- Revise eliminate questions based on respondent & interviewer comments

- Pilot test again, if necessary

- Prepare final interviewer instructions

- Be prepared to modify questionnaires if interviewer training raises problems

- After interviewing is complete, debrief interviewers for potential problems

- Use experience from one study for future planning

10. How to draft and organize your questionnaire:

Ordering the Questions

Decide on order of items/questions

Easy ® difficult

General ® particular

Factual ® abstract

Where to place sensitive questions?

Group questions by topic/ response options

Starting questions

Simple

With closed format

Relevant to main subject

Non-offending

Neither demographic nor personal questions

Don’t put most important item last

What Is A Good Question?

§ One that yields a truthful, accurate answer

§ One that asks for one answer on one dimension

§ One that accommodates all possible contingencies of response

§ One that uses specific, simple language

§ One that has mutually exclusive response options

§ One that produces variability in response

§ One that minimizes social desirability

§ One that is pretested

Language of Questions

a) Ask one question at a time

For example:

Bad question:

In the past 6 months, what major appliances has your household purchased new from the store?

Better question:

Now I’m going to read a list of household appliances. As I read each one, please tell me whether or not your household has purchased this type of appliance new from the store during the past 6 months. How about…

a refrigerator?

a kitchen range or oven?

a microwave?

Bad Question:

Compared to one year ago, are you paying more, less, or about the same for your auto and life insurance?

Better Question:

Compared to one year ago, are you now paying more, less, or about the same for …

a. auto insurance?

b. life insurance?

b) Specify

à Specify who, what, when, where and how.

For example, whose income? What’s included? Over what period of time? Example:

“In 2002, what was your total household income, before taxes? Please count income from all members of your household, including wages from employment, disability, social security, and public aid.”

c) Specify through cues

Example

People drink beer in many places – for example, at home, at restaurants, at bars, sporting events, at friends’ homes, etc. During the past 30 days, did you drink any beer?

d) Use words with singular meanings

Ambiguous:

How would you compare how close you are to family in your current neighborhood to how close you were in your old neighborhood? Would you say your family is closer here, further here, or the same?

More Clear:

Compared to your last neighborhood, do you now live closer to your family, are you further from your family, or are you about the same distance?

e) Be appropriate

Are you a drunk?

- Yes 

- No 

Vs.

How often have you consumed alcoholic beverages during the past 6 months?

- Daily 

- 2-6 times/week 

- Once a week 

- Less than once a week 

- Don´t know 

f) Be simple

Did you smoke not less than a mean amount
of 7 cigarettes/2 days from 1999 onwards?

· Yes 

· No 

Vs.

Did you smoke an average of 2 pack of cigarettes/week for the last 5 years?

· Yes 

· No 

· Don´t know 

g) Be accurate

Do you often touch dogs?

· Yes 

· No 

Vs.

How often did you touch a dog during the
past 3 months?

· Once 

· Twice 

· Three times or more 

· Not at all 

· Don’t know 

h) Question Placement

· It's a good idea to put difficult, embarrassing or threatening questions towards the end of the interview when the interviewee has gotten more comfortable. This has two benefits. First, it makes them more likely to answer, and, second, if they get mad and leave, at least you've gotten most of your questions asked!

· Put related questions together to avoid giving the impression of lack of meticulousness

i) Filtering "Don't Know"

There are three ways of dealing with "don't know". Standard format. No "don't know" option is presented to the respondent, but is recorded if the respondent volunteers it. Quasi filter. A "don't know" option is included among the possible responses. Full filter. First the respondent is asked if they have an opinion. Then, if yes, they ask the question.

j) Social Desirability

Ø Respondents will try to represent themselves to the interviewer in a way that reflects positively on them

Ø As questions become more threatening, respondents are more likely to overstate or understate behavior, even when the best question wording is used.

Types of Questions

è Two main question formats:

Closed format ® forced choice

Yes  Always 

No  Sometimes 

Don’t know  Never 

Open format ® free text

What is your most distressing symptom? Please describe: ____________________________________________________________________________

Closed Questions

1. Straightforward response

What is your age in years? ___ years

How long have you owned a dog? ___ years

What is your sex (gender)?

Male 

Female 

Did you stay in Hotel X on 25/11/08?

Yes 

No 

Don’t know 

2. Checklist

Which of the following outdoor activities did you do last week?

Running 

Walking 

Hiking 

Cycling 

Swimming 

3. Rating scale

Did you do use sunscreen during the following outdoor activities during the past six months?

Always Sometimes Seldomly Never

Running    

Walking    

Cycling    

4. Rating scale Numerical

How useful would you think that information on the risk of biting from stray dogs would be?
(Please circle)

1 2 3 4 5 6 7

Not at all useful Very useful

5. Scales for measuring attitude (Likert)

Stray dogs carry a higher risk of rabies

No, I strongly disagree 

No, I disagree quite a lot 

No, I disagree just a little 

I’m not sure about this 

Yes, I agree just a little 

Yes, I agree quite a lot 

Yes, I strongly agree 

------------------------------------------------------------------------------------------

Advantages

Disadvantages

Closed-ended

· Easy and quick to answer

· Answers across resps easy to compare

· Answers easier to analyze on computer

· Response choices make question clearer

· Easy to replicate study

· Can put ideas in resp's head

· Resps w/ no opinion answer anyway

· Resps can feel constrained/frustrated

· Many choices can be confusing

· Can't tell if resp. misinterpreted the question

· Fine distinctions may be lost

· Clerical mistakes easy to make

· Force respondents into simple responses

Open-ended

· Permit unlimited number of answers

· Resps can qualify and clarify responses

· Can find the unanticipated

· Reveal resps thinking processes

· Resps give answers w/ diff. level of detail

· Answers can be irrelevant

· Inarticulate or forgetful resps are at disadvantage

· Coding responses is subjective and tedious

· Requires more resp. time and effort

· Intimidates respondents

· When resp omits a response, can't tell if its because
of belief or just forgetfulness

Contingency Questions

These are questions of the type:
"If you have done this…. Go to question number such and such…".
In this case you are telling your respondents to skip some questions that do not apply to them and go to another section.

Example:
Was your child breastfed for the first three months after birth?
Yes [ ] No [ ]
If NO please go to questions 25-30 on page 4.

12. Problems and Pitfalls

Avoid questions that ask two things at once - you won’t know which ‘bit’ people are answering:

Have you ever had stomach ache and diarrhoea?

Ambiguity.....

Do you go to the woods a lot?

Avoid jargon/abbreviations/slang

How often do you get up at night to PU? (pass urine)

Should IVDUs be treated in the community?

Avoid not mutually exclusive options

What age are you?

16-20 

20-25 

25-30 

35-40 

Avoid leading questions

Do you think that the food in the hotel made you sick?

Did the hotel staff seem unhygienic to you?

Do you agree that the hospital staff were close to exhaustion?

Avoid emotional language, prestige bias and leading questions

o "What should be done about murderous terrorists who threaten the freedom of good citizens and the safety of our children?"

o "Most doctors say that cigarette smoke causes lung disease for those near a smoker. Do you agree"?

o Avoid leading questions like "You don't smoke, do you?" or "I assume you would agree that the teachers do a heroic job for our children".

o Avoid loading questions with extra adjectives and adverbs, like "Should the mayor spend even more tax money trying to keep the streets in top shape?"

Avoid asking questions beyond a respondent's capabilities

o People have cognitive limitations, especially when it comes to memory of past events. Asking "how did you feel about your brother when you were six years old" is probably useless.

o It is pointless to ask people about things that are not natural ways for them to think. For example, don't bother asking "How many gallons of gasoline did you buy for your car last year?”

Avoid false premises

Asking "What is the most important thing we should do stop the economy from deteriorating any further?" assumes that the economy is deteriorating, which the respondent may not agree with. This puts the respondent in a tough spot. It would be better to rephrase as "What is the most important thing a government can do to strengthen its economy".

Avoid asking about future intentions (if you can) Hypothetical questions like "If a new grocery store were to open down the street, would you shop there?" are notoriously unrelated to actual future behavior.

Avoid negatives and especially double negatives

o Negatives like "Students should not be required to take a comprehensive exam to graduate" are often difficult for many respondents to process, especially if they agree with the predicate, because then they are disagreeing with not doing something, which is confusing!

o Double negatives like "It is not a good idea to not turn in homework on time" yield very unreliable data because people are unsure about whether to put a "yes" or "no" even if it is clear in their minds whether turning homework in on time is a good idea.

Typographical / spelling errors

Format of Questions

13. Physical Format of the Self-Administered Questionnaire

Careful formatting is necessary to decrease errors and increase motivation

Respondent’s needs must always take priority, followed by interviewer and data processors

The questionnaire should be uncluttered and spread out. Do not skimp on paper. It is better to have a clearly structured questionnaire on several pages than a tightly packed one which is difficult to read.

Chose a clear format for the respondents to record their answers.

Examples: · Tick boxes - Usually when there is a choice of possible answers you should provide either brackets [ ] or parentheses ( ) for respondents to record their answers. · Circle items - Alternatively you can provide the answers already precoded so that respondents only have to circle the correct answer. Pre-coded answers have the advantage that you can use those codes later in the data-processing stage.

14. Physical Format Checklist

Number all questions sequentially

Use large, clear type; don’t crowd

‘White space:’ Place more blank space between questions than between subcomponents of questions

List answer categories vertically instead of horizontally

Avoid double/triple ‘banking’ of response choices

Be consistent with direction of response categories

Be consistent with placement of response categories

Don’t split questions across pages. If necessary (e.g., question requires 1.5 pages), restate question and response categories on next page

Put special instructions on questionnaire as needed, next to question

Distinguish directions from questions

Precode the questionnaire (vs. check boxes)