Everyone’s got an opinion about candidate satisfaction survey bias: Surveying newly hired employees is positively biased. Surveying rejected candidates is negatively biased. Surveying in real time results in survey fatigue and positive bias because candidates tell you what you want to hear.
They are all wrong and they are all correct. This article looks at three common approaches to surveying and explores pros and cons.
Here’s a breakdown of the three most common approaches.
Surveying Newly Hired Employees
Many organizations survey new hires about their candidate experience. If reducing candidate satisfaction survey bias is your aim, do not use this approach! There are a couple basic reasons for this. The first being that only a very small fraction of candidates get hired. So right out of the gate, you are narrowing the kind of feedback you’ll get and limiting the statistical validity of your data. The fewer responses you get the less valid any conclusions you can draw. This is statistics 101.
The second reason that surveying newly hired candidates is a bad idea, beyond the fact that you are missing feedback from MOST of your candidates, is that you are asking the winners to critique the game. Of course you will get positive candidate survey bias unless your hiring process selects for iconoclastic employees who think nothing of starting a new job and criticizing the process that got them there at the same time.
As one Survale client put it, “We wanted to know the good, the bad and the ugly of our candidate experience so we could optimize our processes around candidate satisfaction.”
Unfortunately, that is not possible through new hire surveys.
Pros:
- You feel good about getting high Net Promoter Scores
- Surveying can be accomplished with basic survey tools
Cons:
- Only gives you feedback from a tiny fraction of your candidates
- Huge positive bias (confirmed by Survale client data)
Surveying All Candidates When Requisition Closes
Many who believe real time feedback results in positive candidate survey bias swear by waiting and then surveying all candidates after a requisition closes. By waiting until the end of the process, you get the true feelings of rejected candidates and you are surveying the largest pool available so you get a broad set of respondents.
However, it can take weeks to close a requisition. Oftentimes it takes months before a req is fully closed within your ATS. And, of course, it’s not unheard of that reqs never get closed due to poor ATS user discipline.
This means that surveying when the req closes can suffer from lack of recency. You are asking candidates, well after the fact, to remember everything they experienced from application experience to phone screen to interview, etc..
Add to this the fact that the experience of being rejected at one particular stage can negatively impact a candidates perception of their experience at previous stages. Waiting until the end misses an opportunity to get uncolored feedback on all stages of the hiring process.
This approach also suffers from the lower response rates because candidates aren’t super motivated to answer questions for you after rejection (unless a particularly heinous experience is deeply etched in their mind).
Pros:
- Surveying all candidates results in reduced positive bias
- Can be accomplished with basic, unsophisticated survey tools
- Better suited to broad candidate experience benchmarking
Cons:
- Reduced response rates
- Responses less reliable because it relies heavily on candidate recollection
- Not a great approach if you want to accurately analyze feedback at each recruiting stage
Real Time Feedback – Removes Candidate Survey Bias
The real time feedback approach automatically collects survey data after each step of the hiring process. From career site to application to phone screen to interview to offer, etc.. As the candidate finishes one step and they are statused in the ATS to the next one, or they are rejected, surveys are triggered to ask these candidates about that step.
This approach results in higher response rates because you are surveying candidates as they are engaged in the process and motivated to be responsive, not after. The feedback is most accurate because their memories are fresh. And because they are being surveyed on a single stage, candidates have fewer questions to answer so they are more likely to respond.
The best part of real-time feedback is that it is by definition the most complete and unbiased way to gather feedback. With this approach, organizations get feedback from candidates both when they are in-process and engaged AND when they get rejected. This blend of positive and negative bias dilutes and reduces both forms of bias.
This is because the req ends for recruiters when a candidate is hired. But the req ends for each candidate when they are rejected. Gathering feedback in real time captures the same “end of req” feedback as waiting until the req officially closes, but it gets it in real time for each candidate as their journey comes to an end.
Real time surveying makes for the most complete and accurate data set, with the least bias. It combines the in process and engaged candidate feedback, as well as the hired and rejected candidate feedback into a heterogeneous pool. And it provides their satisfaction data at each point of the process. Any positive bias that is introduced by surveying candidates when they are still in the running is tempered by the potential negative bias introduced by surveying candidates after they’ve been rejected.
And one other thing: Surveying in real time allows for taking action to fix problems in the hiring process much sooner. Many Survale clients use real-time feedback to turn unsatisfied candidates around and tweak the people, processes and technologies causing dissatisfaction.
If you want to use an approach that yields more of the most useful kind of feedback and reduces candidate satisfaction survey bias, gathering candidate feedback in real time is truly the best of all worlds.
Pros:
- Minimizes potential negative and positive biases
- Results in higher response rates (more data)
- Provides more accurate feedback because of less reliance on memory
- Enables accurate analysis of sentiment at each stage of the process
- Data is gathered automatically in the background with no manual surveying required
Cons:
- Not all survey tools are capable of this type of approach
