Home » Part D: Methodological background of data collection » D.6 Step 6: Choice of data collection method

D.6 Step 6: Choice of data collection method


Please consider: The choice of the right data collection method is closely related to the questionnaire design. Thus, although here the choice of the data collection method is presented as a step preceding the questionnaire design, they have to be considered together.

When you have decided how many times you want to collect organic market data, and whether you want to collect it from exactly the same or just comparable elements, you should now focus on the way you want to collect it. For example, depending on your aim and the type of data you want to collect, in some cases calling people on their phone might be a good option. In other cases you might decide to do a survey by sending them an email with a link to your questionnaire. The most important data collection methods are presented in the figure below (Figure D.6-1). In market data collection, electronic formats (online and mail surveys as well as computer assisted telephone- and face-to-face surveys) are currently the most important survey modes.

D.6.1 Online surveys

There are two ways through which respondents can be contacted in an online survey: Either a link to a questionnaire is sent via e-mail (e-mail survey), or such a link is based on a website (web/ internet based survey). In online surveys, no interviewers are involved. Hence, the questionnaire needs to be self-explanatory and to include very clear instructions, questions and response categories. A lot of attention should be paid to an attractive and easy-to-fill-out questionnaire format.

A great disadvantage of online surveys is that the response rates are usually very low (mostly < 5%). This can lead to biased results as those people who do respond are normally more interested in the topic than the nonrespondents. Additionally, an online survey is only accessible by people who own a computer and know how to use it. Another disadvantage of online surveys is that you cannot be completely sure who is actually filling in the questionnaire. Furthermore, technical problems need special attention to avoid data loss.

However, in some cases the advantages of online surveys can outweigh the problems connected with them. First of all, as no interviewers are involved, this is one less source of error. Second, data from online surveys are immediately available in an electronic format. This saves time and money for manually entering data, and helps to avoid another source of error. Additionally, the extent of the questionnaire (and thus the number of questions that need to be read by each respondent) can be reduced by the introduction of automatic filters (see the next part on questionnaire design for more information on filter questions). Answer inconsistencies can be detected through automatic plausibility checks. Online surveys are quite cheap to conduct as you do not have to pay interviewers, travel costs or costs for stamps etc. (such as in mail surveys).

If your population of interest has access to computers and knows how to use them, and if there is experience with online surveys in your organisation, they can be a reasonable way to collect certain types of market data. In order to increase response rates, it can be a good option to combine an online survey with follow-up telephone calls to remind the participants of their participation (Lozar et al.  2008) and/or to offer participants incentives. For a survey on organic product prices among farmers, you could for example offer to send the survey results to those people who answered the complete questionnaire. Knowledge about these prices will provide a benefit for farmers, so this incentive could motivate them to participate in the survey.

To top

D.6.2 Mail surveys

In mail surveys, a (printed) questionnaire is sent to people of a certain population via ‘ordinary’ mail (not e-mail). The envelope in which it is sent should further contain a return envelope for sending the filled-in questionnaire back, with the return address and sufficient postage (freepost) provided.  With mail surveys, you can also access people who do not have a computer or internet access. Otherwise, in many ways mail surveys are very similarly to online surveys, and you have to consider the same issues in their design.

The response rate of mail surveys, though often higher than in e-mail or online surveys, is usually still quite low, very often between 5 and 10% only. Furthermore, you obviously need to know the postal addresses of all elements of your population of interest. Even if you have to pay for printing costs as well as for envelopes and postage, mail surveys are still a cheaper option than personally conducted interviews. Participants of both mail and online surveys can fill in the questionnaires whenever it suits them, so the time pressure is reduced.  In addition, respondents have more privacy and there is no interviewer bias, so that more honest answers can be expected (de Leeuw & Hox, 2008).

D.6.3 Telephone surveys

As telephone surveys are conducted by interviewers, they have the potential problem of interviewer bias (the influence on a respondent through an interviewer). Hence, it is very important that not only the questions, but also the interviewing procedure are standardized and precisely followed by all interviewers. Telephone surveys should also be rather short, and the questions need to be very clear.

Response rates for telephone surveys are also quite low, but usually slightly higher than for online or mail surveys. It might be difficult to reach people of the population of interest, either because they are not available (nonresponse) or because not all telephone numbers are available (undercoverage). Mailing out a letter informing about the survey in advance of contacting by phone may improve response rates. Telephone surveys, however, have lower costs and less interviewer bias than face-to-face interviews. Compared to mail and online surveys, respondents can ask for clarification in telephone surveys. Telephone surveys can be carried out with the assistance of computers (so-called computer assisted telephone interviews, CATI). One basic function of CATI is that during the interview the interviewer directly enters the respondent’s answers into an electronic format (Steeh, 2008). Specialised marketing agencies further use computers for the automatic creation of telephone numbers (as address books are normally incomplete) and for automatic dialling. In this case, contacted numbers are only connected with an interviewer if the phone call is answered by a person and not by an answering machine.

To top

D.6.4 Face-to-face surveys

Face-to-face surveys are rarely used in market data collection, as it might not be feasible to approach all respondents personally; especially if they are widespread in larger geographical areas. In face-to-face surveys the respondents are directly questioned by the interviewer, so that interviewer bias can be problematic. Furthermore, face-to-face surveys are very costly and time-consuming. Their major advantage is that the response rates are relatively high. In addition, the questionnaire can be longer and more complex. Face-to-face surveys can also be conducted in a computer-assisted manner, so that the structure and interview procedure is provided on the screen. If the interviewer guides through the survey and uses the computer for assistance, these interviews are referred to as CAPI (computer assisted personal interviews). If interviewees conduct the survey themselves, following the instructions on the screen, the interviews are called CASI (computer assisted self-interviews). In CASI the respondents fill in the questionnaire on the screen without any assistance and guidance through an interviewer. This way, it is possible to reduce the interviewer bias (Loosveldt, 2008). Data are available in an electronic format immediately after the survey.

D.6.5 Mixed-mode survey

Mixed-mode surveys refer to surveys, in which two data collection methods are combined. There are two different options. On the one hand groups of participants can be contacted in different ways (parallel data collection) and on the other hand groups of participants can be approached with two different types of surveys – one after the other (sequential).  Depending on the accessibility of respondents, the combination of two types of surveys might increase the response rate.

The checklist contains the relevant questions for your choice of the right research design.

Checklist: Which is the right data collection method for you?

  • Which data collection method have other people used for the same kind of data collection?
  • What contact information do you have for the elements in your sampling frame (e.g. e-mail or postal addresses)
  • For mail and e-mail surveys: Do all members of your population of interest have access to a computer and know how to use it?
  • What is your sample size?
  • How much nonresponse can you accept?
  • How much money do you have?
  • How much time do you have?

To top