Aviv Raizman,
Customer Success Manager at Dooblo
Estimated reading time: 7 minuets
Main topics
- What is Straightlining and how it effects our data quality
- The benefits of detecting straightlined results on site
- Suggested add-ons during the interviewing process
Kurtosis Research* is a (fictional) market research company, which has just finished retail audit fieldwork, conducted at several points of sale.
In a self-administered survey, respondents completed a 15 closed question interview in which they were asked about their overall shopping experience from their local supermarket, as well as some specific store parameters.
After analyzing the project’s data, the research team noticed several patterns of responses: a significant number of respondents gave identical scores to most questions, no correlativity between the general satisfaction rate to the different parameters, and an overall fast response rate (as opposed to the expected time evaluated for the survey).
This article aims to deconstruct the scenario experienced by Kurtosis Research: what drew the response pattern, how we can prevent such trends and how can you detect similar scenarios in our own projects.
Our fieldwork environments embody multiple characteristics, effecting respondents’ process of decision making. Arbitrarily, we can divide such factors into two groups: external and internal.
While internal factors consist of personal features of our project’s respondents (age, education, sector, etc.), external factors can be found in the setting of our fieldwork: where its being conducted (Indoors, outdoors, in a mall, or perhaps at someone’s home?) when (during morning rush-hour or on a mellow afternoon?) and with whom (in a personal interview handled by a surveyor, a self-administered questionnaire or maybe both?).
Regarding the Kurtosis project, we know several of these factors: the survey was conducted indoors and was self-administered by the respondents. Despite the lack of a specific timeframe for when the interviews were completed, we do know many of the respondents wanted to complete their interview as fast as possible.
A frequent behavior among a respondent’s set of actions during a survey, “Straightlining” is the pattern of sequential response where one answers questions repetitively without paying attention to the question content. In most cases, this phenomenon is the result of boredom and/or time restrictions, resulting in the respondent’s wish to finish the survey quickly. Possibly in order to please the surveyor or in order to receive an incentive without making a significant effort.
Detecting Straightlining and Its Significance
The quality level of our data is the core component of a successful project, and our aim is to achieve quality data without investing significant amounts of resources.
It is important to focus our respondents to a point where they provide meaningful input, whether the answers are open or closed.
SurveyToGo contains built-in quality control features to inspect the quality of collected data: capturing audio, photos and videos throughout the interview session, as well as flagging which is designed to mark irregular results – including duration measures of the response process. These features are available for inspection after the session is complete, and in most cases will require personal supervision to review detected cases.
Regarding the scope of human resources dedicated to review the data, we can also ensure the quality level of the collected data in real-time.
With the solutions suggested later in this article, we are able to shorten the time spent on data revision and the overall time dedicated to our research project using a second protection mechanism powered throughout the data collection process.
Modifying the solutions
Our main challenge is to characterize the sequences of Strightlining. In other words, where in the survey respondents might lose focus and avoid a meaningful response?
It is recommended to integrate several strategies throughout the survey, in order to avoid Straightlining. In our current scenario, we can understand most of the fieldwork’s challenges were related to the duration: some respondents engaged in the interview for a short amount of time, which in some cases resulted in Straightlining and noncorrelative results. In order to prevent these patterns, we recommend considering the following features:
1. Duration Settings
Setting a minimum duration for each question will prevent respondents from responding too quickly. Through the question’s “Advanced” tab, you can set the minimum time required to answer an individual question.
This is an additional feature to quality control flagging of the overall survey duration, which can be set in the general Quality Control menu of the survey:
2. Straightlining Mechanism
After scripting the survey’s questions, you can create a mechanism that will review the respondent’s input. e.g., in a series of 10 single choice questions in the scale of 1-5, a subject has given the score of 5 to each of the series’ questions. In this case, we can create a dummy variable that will consist of a script that reviews the subject’s input. In this case, we would like to inform the respondent about his input using a prompted notification. Later, using jump rules, the respondent can review his answers and modify them after reevaluating his input.
3. Relationship Mechanism
During a Straightlining phase, noncorrelation can often occur when there’s an overall satisfaction question. e.g. – a respondent who rated all the tested parameters with high scores, but gave a low score to the general satisfaction question.
Either when appearing at the start or the end of the interview, the weight of the overall satisfaction question demands special attention from the respondent, which may differ from the scope of attention dedicated in the majority of the interview. Hence, a mechanism that sums the relationship between the general satisfaction question and the rest of the tested parameters may inform the respondent of his/her logical pattern.
Conclusions: Ensuring Your Project’s Data Quality
In this brief article we reviewed the scenario of Straightlining: what causes this response pattern, its significant impact on our data and how to avoid it using scripting mechanisms.
Unlike quality control processes managed after the data has been collected, using the suggested solutions can save a great deal of resources invested in the post-production stages, allowing respondents to review their own response pattern.
The characteristics of your own projects are unique and as mentioned, impacted by both internal and external factors. Before making decisions on which strategies you would like to apply, you should consider several questions:
What is the length of the survey? We wouldn’t want to significantly extend the duration for the respondent’s review.
Who is the target audience? Some strategies might be unpleasant for certain age groups and affect their responses (e.g., a time constraint for a question in a survey among children).
How is the survey managed? In the case of a personal interview, we need to decide if we want to observe the surveyor, or the respondent. Unlike personal interviews, self-administered surveys contain “soft supervision” of the surveyor and accordingly, the strategy involved should be completed by the respondents.
We recommend reviewing these questions prior to the implementation of any of the suggested techniques, in order to ensure the right flow of your project and its quality of data.