Managing raw data and building data quality checks

In Cxoice, all data is collected as an auditable data-stream. The data is captured, and then stored without overwriting previous data, so if repeated attempts are made to answer questions all of the responses are captured. Data capture can be extended to collect mouse movements and keystrokes for monitoring purposes. This helps prevent fraud where click-farms or bots attempt to work through a survey for incentive rewards.

 

The data-stream is translated into different data formats, as required, by pulling data extracts. However, some phone units like to correct spellings and update open ends at the end of a call. This type of data editing can be done by working through a previously completed survey and updating the contents. Note that the original data is still maintained, as the ‘current’ dataset is updated.

 

Quality Scoring

 

The raw data views in the survey monitor show a technical view of the currently captured data and can be copied and pasted into a spreadsheet and used to generate Quality Score ratings per respondent.

 

A Quality Score rating should be calculated by checking the raw data for punching patterns, poor quality answers, inconsistencies and other quality checks. Automated assessments can be used, but we recommend a custom Quality Score calculation sheet is generated for each survey as many quality checks will be dependent on the questions asked and logical consistencies between questions that are not possible for automated checking.

 

The principle behind a Quality Score system is to give demerits for different types of fault within the questionnaire. We don’t expect perfect completion as human beings make errors and edge cases are common when sampling across a large population. Instead we would rate faults using 1, 10, 100, 1000 and 10000 depending on severity, ensuring every participant – completes and non-completes are scored (punching patterns sometimes show up because the same person attempts the survey several times in a go).

 

By using a score, rather than straight rejection, we can use filtering on the data set by Quality Score both to set thresholds for what is accepted, but also to do analysis by quality score, so as to be able to look for systematic anomalies created by poor quality responses.

 

Data extracts, coding and data editing

 

A data extract can be drawn from the data-stream at any point during the survey, and then used for reporting using Cxoice’s reporting options for tabulations, charts and dashboards. Data extracts can pull data from across multiple surveys, both for large projects running suveys together, but also across projects for comparative or time-based data.

 

Each data extract can be viewed and then marked up with edits, or worked with offline to create open-ended coding and edits that can then be reapplied to any future data extracts via an edit file. (See reporting and data extracts)

 

The use of edit files, rather than adjusting the data directly means that data auditability can be maintained and that original data is retained, with the processes used for editing and cleaning that data are transparent and on top of the original data without deleting it.