9/19/2017

Three Lessons Google Taught Us on People Analytics

People in Google

To understand what Google has brought to data science in HR, Work Rules!: Insights from Inside Google That Will Transform How You Live and Lead, by Laszlo Bock, is essential reading. Or at least read this post, be credulous, and believe what I’m telling you.

When Laszlo Bock published his book in 2015, he was still Google’s Senior Vice President of People Operations. In the book, he talks about his experience as the head of a department that went from 6,000 employees, when he started working for Google in 2006, to almost 60,000.
In 2007, hiring reached its highest point of 200 new employees every week.
In 2015, Google was already receiving two million job applications

Lesson 1. Google invented the name of "people analytics"

If today we call it ”people analytics” instead of ”HR Analytics, ”talent analytics,” or ”workforce analytics,” it’s Google’s fault.
They (Google) rechristened their HR department as ”People Operations,” and the organization’s people-centered analysis is called ”people analytics.” The name stuck. ”People Operations” has also become a popular name for HR departments: Dropbox, Facebook, LinkedIn and more than twenty companies have adopted the name.
If you look at the distribution of ”people analytics” vs. ”HR analytics” in Google Trends (09/26/2017), we see that "HR analytics" is searched twice as often as ”People analytics”. HR is predominant in all major countries, except for Brazil.
Note: Thanks, Anand K. Chandarana, for warning that using quotation marks when searching Google Trends. Results change dramatically.





Lesson 2. Algorithms for Recruiting

As Laszlo Bock recognizes, ”for Google, recruiting is more important than training.
At Google, we spend more than double than the average company in recruiting. If we get better at choosing people, that means we’ll have less work to do with them once they’re hired.
Google’s People Analytics workgroup decided to compare the interviewers’ predictions with the new hires’ actual performance to find out how well the interviewers could predict performance.
The results were disappointing. Despite all the time and effort, the study showed that the interviews didn’t predict future performance very well.
From the very first moment, the interviewer knew subconsciously if they liked the candidate or not. From there, and for the next hour, they looked for signs that confirmed their first impression.
Ultimately, if a candidate left a bad first impression, it was almost impossible to turn it around.
Google decided to tackle the problem by eliminating human bias as much as possible. They did it by automating the interview process. In the interviews they do now, they use an application called “qDroid” that guides the interview.
The interviewer enters the candidate’s job functions, and the application generates the questions. QDroid formulates the questions.This method ensures that the interview is structured.
In addition, because the interviewers don’t make up their own questions, the interviews are less biased. The qDroid questions have been widely tested to accurately predict the candidate’s job performance.

Daniel Kahneman: An Algorithm or at Least a Structured Interview

The conclusions reached by Laszlo Bock’s team are the same as Kahneman’s, the psychologist who won the Nobel Prize in economics for integrating aspects of psychological research into economics, especially with respect to decision-making. After 60 years of research and on-the-job experience in recruiting, Kahneman is clear: an algorithm systematically improves the results of an unstructured interview.
Kahneman reminds us that studies in the area are conclusive: Using algorithms is the most effective method to make recruiting decisions.


The human brain spontaneously makes numerous mistakes when making decisions. One source of errors is, of course, the innate biases that appear to be universal in humans. But above all, states Kahneman, the error is simply due to the extraordinary variability of the process.
In other words, it depends on which side of the bed the interviewer or the candidate got up on that day.

Lesson 3. How many interviews are needed to hire a ”Googler”? The rule of Four

Google’s hiring process was designed to be meticulous in avoiding false positives or toxic hires. They thought that the more Google employees participating in the interviews, the better the hiring decision. That would mean that someone applying to work at Google would be subject to more than a dozen interviews. Such a great effort certainly demands a massive amount of time from employees and could be exhausting for the candidate. The People Analytics team decided to study the data to see what part of an interview is really useful.
At Google, interviews are done one-on-one. Each interviewer rates the candidate. Looking at the data, they proved that, indeed, as opposed to looking at a single interview score, the average score of two interviews were a far better indicator of whether a candidate would receive an offer. What happens to the predictive accuracy if we add a third interviewer’s score? Or a fourth one? It turns out that there’s a diminishing return in an interviewer’s feedback. Based on their statistical analysis, the team found that four interviews were enough to predict if someone would be hired at Google with 86% reliability.
After the fourth interview, the increase in precision dropped dramatically with each additional interview. The accuracy in calculating the likelihood of the candidate being hired rose less than one percent.

This graphic shows the increase in the accuracy of the average interview score with each increase in the number of interviewers. Four interviews were enough to predict a new Google employee’s performance with 86% reliability. After the fourth interview, the accuracy of the average score increases less than one percent. The implications of this investigation were obvious: They could make hiring decisions confidently with only four experienced Google employed interviewers. Armed with this data, the Google People Analytics team proposed what they called the ”Rule of Four,” which they use in standard hiring. The Rule of Four reduced the average hiring time to about two weeks, saved the employees hundreds of thousands of hours of interview time, and helped reduce the already stressful process for the candidates.